* crash while playing bzflag
@ 2007-09-07 19:56 Alex Riesen
2007-09-07 23:14 ` Chuck Ebbert
2007-09-07 23:18 ` Michal Piotrowski
0 siblings, 2 replies; 6+ messages in thread
From: Alex Riesen @ 2007-09-07 19:56 UTC (permalink / raw)
To: linux-kernel
Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
Ubuntu Feisty, Radeon R200 (9200) dual head, MergedFB, BZFlag in
OpenGL mode, frozen. That'll teach me playing games at home...
BUG: unable to handle kernel paging request at virtual address ffa85000
printing eip:
c016eed1
*pde = 00005067
*pte = 00000000
Oops: 0000 [#1]
PREEMPT SMP
Modules linked in: binfmt_misc nfs radeon drm nfsd exportfs lockd sunrpc fan firmware_class it87 hwmon_vid hwmon p4_clockmod speedstep_lib ipv6 sg sr_mod cdrom usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_pcm snd_mixer_oss snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device generic floppy snd ide_core intel_agp e100 uhci_hcd ehci_hcd soundcore snd_page_alloc agpgart evdev
CPU: 0
EIP: 0060:[__link_path_walk+2146/2867] Not tainted VLI
EFLAGS: 00010287 (2.6.23-rc5-t #138)
EIP is at __link_path_walk+0x862/0xb33
eax: ffa85000 ebx: f0a4dd64 ecx: c0442130 edx: c1782d00
esi: eee51f30 edi: ffa85000 ebp: f0e49e40 esp: eee51de4
ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068
Process command-not-fou (pid: 2895, ti=eee51000 task=f08f1020 task.ti=eee51000)
Stack: f474c02c 00000101 f1db2d64 c016d3eb c1782d00 ffa85000 00000000 00000000
00000000 96ba5598 0000000b f474c021 c18eff00 f0e49e40 f08e1540 eee51f30
c1937c78 c18eff00 c016f1e6 f474c000 c1937c78 c18eff00 c180b180 f11a7600
Call Trace:
[do_lookup+79/323] do_lookup+0x4f/0x143
[link_path_walk+68/179] link_path_walk+0x44/0xb3
[_spin_unlock+5/28] _spin_unlock+0x5/0x1c
[get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
[do_path_lookup+362/463] do_path_lookup+0x16a/0x1cf
[__path_lookup_intent_open+69/117] __path_lookup_intent_open+0x45/0x75
[path_lookup_open+32/37] path_lookup_open+0x20/0x25
[open_namei+114/1364] open_namei+0x72/0x554
[unmap_vmas+791/1240] unmap_vmas+0x317/0x4d8
[do_filp_open+37/57] do_filp_open+0x25/0x39
[_spin_unlock+5/28] _spin_unlock+0x5/0x1c
[get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
[do_sys_open+68/192] do_sys_open+0x44/0xc0
[sys_open+28/30] sys_open+0x1c/0x1e
[sysenter_past_esp+95/133] sysenter_past_esp+0x5f/0x85
[xfrm_bundle_ok+53/522] xfrm_bundle_ok+0x35/0x20a
=======================
Code: f0 ff ff 0f 87 38 01 00 00 8b 46 1c 8b 44 86 20 89 44 24 14 31 ff 85 c0 0f 84 09 01 00 00 89 c7 3d 00 f0 ff ff 0f 87 f5 00 00 00 <80> 38 2f 0f 85 9f 00 00 00 89 f0 e8 38 e1 ff ff 64 a1 00 70 3f
EIP: [__link_path_walk+2146/2867] __link_path_walk+0x862/0xb33 SS:ESP 0068:eee51de4
SysRq : Emergency Sync
Emergency Sync complete
SysRq : Emergency Sync
Emergency Sync complete
The config, lspci output, Xorg.0.log, and a more complete log of the
crash attached (the crash happened around Sep 7 21:24:27 in the log,
I panicked a bit and pressed Alt-SysRq-t and emergency sync).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: crash while playing bzflag
2007-09-07 19:56 crash while playing bzflag Alex Riesen
@ 2007-09-07 23:14 ` Chuck Ebbert
2007-09-08 18:11 ` Alex Riesen
2007-09-07 23:18 ` Michal Piotrowski
1 sibling, 1 reply; 6+ messages in thread
From: Chuck Ebbert @ 2007-09-07 23:14 UTC (permalink / raw)
To: Alex Riesen, linux-kernel
On 09/07/2007 03:56 PM, Alex Riesen wrote:
> Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
> Ubuntu Feisty, Radeon R200 (9200) dual head, MergedFB, BZFlag in
> OpenGL mode, frozen. That'll teach me playing games at home...
>
> BUG: unable to handle kernel paging request at virtual address ffa85000
> printing eip:
> c016eed1
> *pde = 00005067
> *pte = 00000000
> Oops: 0000 [#1]
> PREEMPT SMP
> Modules linked in: binfmt_misc nfs radeon drm nfsd exportfs lockd sunrpc fan firmware_class it87 hwmon_vid hwmon p4_clockmod speedstep_lib ipv6 sg sr_mod cdrom usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_pcm snd_mixer_oss snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device generic floppy snd ide_core intel_agp e100 uhci_hcd ehci_hcd soundcore snd_page_alloc agpgart evdev
> CPU: 0
> EIP: 0060:[__link_path_walk+2146/2867] Not tainted VLI
> EFLAGS: 00010287 (2.6.23-rc5-t #138)
> EIP is at __link_path_walk+0x862/0xb33
> eax: ffa85000 ebx: f0a4dd64 ecx: c0442130 edx: c1782d00
> esi: eee51f30 edi: ffa85000 ebp: f0e49e40 esp: eee51de4
> ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068
> Process command-not-fou (pid: 2895, ti=eee51000 task=f08f1020 task.ti=eee51000)
> Stack: f474c02c 00000101 f1db2d64 c016d3eb c1782d00 ffa85000 00000000 00000000
> 00000000 96ba5598 0000000b f474c021 c18eff00 f0e49e40 f08e1540 eee51f30
> c1937c78 c18eff00 c016f1e6 f474c000 c1937c78 c18eff00 c180b180 f11a7600
> Call Trace:
> [do_lookup+79/323] do_lookup+0x4f/0x143
> [link_path_walk+68/179] link_path_walk+0x44/0xb3
> [_spin_unlock+5/28] _spin_unlock+0x5/0x1c
> [get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
> [do_path_lookup+362/463] do_path_lookup+0x16a/0x1cf
> [__path_lookup_intent_open+69/117] __path_lookup_intent_open+0x45/0x75
> [path_lookup_open+32/37] path_lookup_open+0x20/0x25
> [open_namei+114/1364] open_namei+0x72/0x554
> [unmap_vmas+791/1240] unmap_vmas+0x317/0x4d8
> [do_filp_open+37/57] do_filp_open+0x25/0x39
> [_spin_unlock+5/28] _spin_unlock+0x5/0x1c
> [get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
> [do_sys_open+68/192] do_sys_open+0x44/0xc0
> [sys_open+28/30] sys_open+0x1c/0x1e
> [sysenter_past_esp+95/133] sysenter_past_esp+0x5f/0x85
> [xfrm_bundle_ok+53/522] xfrm_bundle_ok+0x35/0x20a
> =======================
> Code: f0 ff ff 0f 87 38 01 00 00 8b 46 1c 8b 44 86 20 89 44 24 14 31 ff 85 c0 0f 84 09 01 00 00 89 c7 3d 00 f0 ff ff 0f 87 f5 00 00 00 <80> 38 2f 0f 85 9f 00 00 00 89 f0 e8 38 e1 ff ff 64 a1 00 70 3f
> EIP: [__link_path_walk+2146/2867] __link_path_walk+0x862/0xb33 SS:ESP 0068:eee51de4
> SysRq : Emergency Sync
> Emergency Sync complete
> SysRq : Emergency Sync
> Emergency Sync complete
>
> The config, lspci output, Xorg.0.log, and a more complete log of the
> crash attached (the crash happened around Sep 7 21:24:27 in the log,
> I panicked a bit and pressed Alt-SysRq-t and emergency sync).
Whee...
here, in __vfs_follow_link:
if (*link == '/') { <================ link points to unmapped memory
path_release(nd);
if (!walk_init_root(link, nd))
/* weird __emul_prefix() stuff did it */
goto out;
}
inlined from __do_follow_link:
if (!IS_ERR(cookie)) {
char *s = nd_get_link(nd);
error = 0;
if (s)
error = __vfs_follow_link(nd, s);
if (dentry->d_inode->i_op->put_link)
dentry->d_inode->i_op->put_link(dentry, nd, cookie);
}
__do_follow_link is inlined from do_follow_link
presumably inlined here:
if ((lookup_flags & LOOKUP_FOLLOW)
&& inode && inode->i_op && inode->i_op->follow_link) {
err = do_follow_link(&next, nd);
if (err)
goto return_err;
inode = nd->dentry->d_inode;
} else
What filesystem was this?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: crash while playing bzflag
2007-09-07 19:56 crash while playing bzflag Alex Riesen
2007-09-07 23:14 ` Chuck Ebbert
@ 2007-09-07 23:18 ` Michal Piotrowski
2007-09-08 18:12 ` Alex Riesen
1 sibling, 1 reply; 6+ messages in thread
From: Michal Piotrowski @ 2007-09-07 23:18 UTC (permalink / raw)
To: Alex Riesen, linux-kernel
Hi Alex,
On 07/09/2007, Alex Riesen <raa.lkml@gmail.com> wrote:
> Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
Is this a post 2.6.22 regression?
Regards,
Michal
--
LOG
http://www.stardust.webpages.pl/log/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: crash while playing bzflag
2007-09-07 23:14 ` Chuck Ebbert
@ 2007-09-08 18:11 ` Alex Riesen
0 siblings, 0 replies; 6+ messages in thread
From: Alex Riesen @ 2007-09-08 18:11 UTC (permalink / raw)
To: Chuck Ebbert; +Cc: linux-kernel
Chuck Ebbert, Sat, Sep 08, 2007 01:14:01 +0200:
> On 09/07/2007 03:56 PM, Alex Riesen wrote:
> > Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
> > Ubuntu Feisty, Radeon R200 (9200) dual head, MergedFB, BZFlag in
> > OpenGL mode, frozen. That'll teach me playing games at home...
> >
> > BUG: unable to handle kernel paging request at virtual address ffa85000
> > printing eip:
> > c016eed1
> > *pde = 00005067
> > *pte = 00000000
> > Oops: 0000 [#1]
> > PREEMPT SMP
> > Modules linked in: binfmt_misc nfs radeon drm nfsd exportfs lockd sunrpc fan firmware_class it87 hwmon_vid hwmon p4_clockmod speedstep_lib ipv6 sg sr_mod cdrom usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_pcm snd_mixer_oss snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device generic floppy snd ide_core intel_agp e100 uhci_hcd ehci_hcd soundcore snd_page_alloc agpgart evdev
> > CPU: 0
> > EIP: 0060:[__link_path_walk+2146/2867] Not tainted VLI
> > EFLAGS: 00010287 (2.6.23-rc5-t #138)
> > EIP is at __link_path_walk+0x862/0xb33
> > eax: ffa85000 ebx: f0a4dd64 ecx: c0442130 edx: c1782d00
> > esi: eee51f30 edi: ffa85000 ebp: f0e49e40 esp: eee51de4
> > ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068
> > Process command-not-fou (pid: 2895, ti=eee51000 task=f08f1020 task.ti=eee51000)
> > Stack: f474c02c 00000101 f1db2d64 c016d3eb c1782d00 ffa85000 00000000 00000000
> > 00000000 96ba5598 0000000b f474c021 c18eff00 f0e49e40 f08e1540 eee51f30
> > c1937c78 c18eff00 c016f1e6 f474c000 c1937c78 c18eff00 c180b180 f11a7600
> > Call Trace:
> > [do_lookup+79/323] do_lookup+0x4f/0x143
> > [link_path_walk+68/179] link_path_walk+0x44/0xb3
> > [_spin_unlock+5/28] _spin_unlock+0x5/0x1c
> > [get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
> > [do_path_lookup+362/463] do_path_lookup+0x16a/0x1cf
> > [__path_lookup_intent_open+69/117] __path_lookup_intent_open+0x45/0x75
> > [path_lookup_open+32/37] path_lookup_open+0x20/0x25
> > [open_namei+114/1364] open_namei+0x72/0x554
> > [unmap_vmas+791/1240] unmap_vmas+0x317/0x4d8
> > [do_filp_open+37/57] do_filp_open+0x25/0x39
> > [_spin_unlock+5/28] _spin_unlock+0x5/0x1c
> > [get_unused_fd_flags+198/208] get_unused_fd_flags+0xc6/0xd0
> > [do_sys_open+68/192] do_sys_open+0x44/0xc0
> > [sys_open+28/30] sys_open+0x1c/0x1e
> > [sysenter_past_esp+95/133] sysenter_past_esp+0x5f/0x85
> > [xfrm_bundle_ok+53/522] xfrm_bundle_ok+0x35/0x20a
> > =======================
> > Code: f0 ff ff 0f 87 38 01 00 00 8b 46 1c 8b 44 86 20 89 44 24 14 31 ff 85 c0 0f 84 09 01 00 00 89 c7 3d 00 f0 ff ff 0f 87 f5 00 00 00 <80> 38 2f 0f 85 9f 00 00 00 89 f0 e8 38 e1 ff ff 64 a1 00 70 3f
> > EIP: [__link_path_walk+2146/2867] __link_path_walk+0x862/0xb33 SS:ESP 0068:eee51de4
>
> Whee...
>
> here, in __vfs_follow_link:
>
> if (*link == '/') { <================ link points to unmapped memory
> path_release(nd);
> if (!walk_init_root(link, nd))
> /* weird __emul_prefix() stuff did it */
> goto out;
> }
>
> inlined from __do_follow_link:
>
> if (!IS_ERR(cookie)) {
> char *s = nd_get_link(nd);
> error = 0;
> if (s)
> error = __vfs_follow_link(nd, s);
> if (dentry->d_inode->i_op->put_link)
> dentry->d_inode->i_op->put_link(dentry, nd, cookie);
> }
>
> __do_follow_link is inlined from do_follow_link
>
> presumably inlined here:
>
> if ((lookup_flags & LOOKUP_FOLLOW)
> && inode && inode->i_op && inode->i_op->follow_link) {
> err = do_follow_link(&next, nd);
> if (err)
> goto return_err;
> inode = nd->dentry->d_inode;
> } else
>
> What filesystem was this?
Presumably ext3 (because it is where command-not-found python script
is), but I usually have the following mounted:
/dev/sda9 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda8 on /home type ext3 (rw)
/dev/sda1 on /media/sda1 type ext2 (rw)
/dev/sda2 on /media/sda2 type ext3 (rw)
/dev/sda5 on /media/sda5 type ext3 (rw)
/dev/sda7 on /media/sda7 type ext3 (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: crash while playing bzflag
2007-09-07 23:18 ` Michal Piotrowski
@ 2007-09-08 18:12 ` Alex Riesen
2007-09-08 23:08 ` Alex Riesen
0 siblings, 1 reply; 6+ messages in thread
From: Alex Riesen @ 2007-09-08 18:12 UTC (permalink / raw)
To: Michal Piotrowski; +Cc: linux-kernel
Michal Piotrowski, Sat, Sep 08, 2007 01:18:10 +0200:
> Hi Alex,
>
> On 07/09/2007, Alex Riesen <raa.lkml@gmail.com> wrote:
> > Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
>
> Is this a post 2.6.22 regression?
>
Can't say yet.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: crash while playing bzflag
2007-09-08 18:12 ` Alex Riesen
@ 2007-09-08 23:08 ` Alex Riesen
0 siblings, 0 replies; 6+ messages in thread
From: Alex Riesen @ 2007-09-08 23:08 UTC (permalink / raw)
To: Michal Piotrowski, linux-kernel
Alex Riesen, Sat, Sep 08, 2007 20:12:13 +0200:
> Michal Piotrowski, Sat, Sep 08, 2007 01:18:10 +0200:
> > Hi Alex,
> >
> > On 07/09/2007, Alex Riesen <raa.lkml@gmail.com> wrote:
> > > Kernel: v2.6.23-rc5+ (b21010ed6498391c0f359f2a89c907533fe07fec)
> >
> > Is this a post 2.6.22 regression?
>
Probably not. I can kill the machine on 2.6.22.6 as good as on
2.6.23-rc5. Even without anything in log: just by trying to connect to
a server with players. Will try to connect a serial console tomorrow,
err... later today.
I saw this in logs once, but the machine was running after the BUG:
Sep 9 00:46:19 steel kernel: agpgart: Found an AGP 3.0 compliant device at 0000:00:00.0.
Sep 9 00:46:19 steel kernel: agpgart: Putting AGP V3 device at 0000:00:00.0 into 8x mode
Sep 9 00:46:19 steel kernel: agpgart: Putting AGP V3 device at 0000:01:00.0 into 8x mode
Sep 9 00:46:19 steel kernel: [drm] Loading R200 Microcode
Sep 9 00:46:27 steel kernel: BUG: unable to handle kernel NULL pointer dereference at virtual address 00000010
Sep 9 00:46:27 steel kernel: printing eip:
Sep 9 00:46:27 steel kernel: f9c09bb2
Sep 9 00:46:27 steel kernel: *pde = 00000000
Sep 9 00:46:27 steel kernel: Oops: 0000 [#1]
Sep 9 00:46:27 steel kernel: PREEMPT SMP
Sep 9 00:46:27 steel kernel: Modules linked in: binfmt_misc nfs radeon drm nfsd exportfs lockd sunrpc fan button firmware_class it87 hwmon_vid hwmon i2c_isa p4_clockmod speedstep_lib ipv6 sg sr_mod cdrom usb_storage snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_pcm snd_mixer_oss snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore generic floppy snd_page_alloc e100 ide_core ehci_hcd uhci_hcd intel_agp agpgart evdev
Sep 9 00:46:27 steel kernel: CPU: 0
Sep 9 00:46:27 steel kernel: EIP: 0060:[<f9c09bb2>] Not tainted VLI
Sep 9 00:46:27 steel kernel: EFLAGS: 00210246 (2.6.22.6 #3)
Sep 9 00:46:27 steel kernel: EIP is at radeon_irq_wait+0xdf/0x138 [radeon]
Sep 9 00:46:27 steel kernel: eax: 00000000 ebx: f6c9dc00 ecx: f4f92000 edx: 00200213
Sep 9 00:46:27 steel kernel: esi: 0000214c edi: ffffdf23 ebp: f4f92f00 esp: f4f92ed0
Sep 9 00:46:27 steel kernel: ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068
Sep 9 00:46:27 steel kernel: Process bzflag (pid: 2616, ti=f4f92000 task=f4d314c0 task.ti=f4f92000)
Sep 9 00:46:27 steel kernel: Stack: f6cae800 f9c14ba8 f4f92f00 00000000 f4d314c0 c0114436 f6c9dd00 f6c9dd00
Sep 9 00:46:27 steel kernel: 0000214c f9c09ad3 00000057 f9c14bd8 f4f92f44 f996811c 082af558 00000000
Sep 9 00:46:27 steel kernel: 0003ddfe f4f92f34 c180e120 f4f92f40 00200246 40046457 f4bf9a80 f6c1dbb0
Sep 9 00:46:27 steel kernel: Call Trace:
Sep 9 00:46:27 steel kernel: [show_trace_log_lvl+26/47] show_trace_log_lvl+0x1a/0x2f
Sep 9 00:46:27 steel kernel: [show_stack_log_lvl+157/165] show_stack_log_lvl+0x9d/0xa5
Sep 9 00:46:27 steel kernel: [show_registers+497/818] show_registers+0x1f1/0x332
Sep 9 00:46:27 steel kernel: [die+272/529] die+0x110/0x211
Sep 9 00:46:27 steel kernel: [do_page_fault+1061/1268] do_page_fault+0x425/0x4f4
Sep 9 00:46:27 steel kernel: [error_code+114/120] error_code+0x72/0x78
Sep 9 00:46:27 steel kernel: [<f996811c>] drm_ioctl+0x154/0x19c [drm]
Sep 9 00:46:27 steel kernel: [do_ioctl+139/163] do_ioctl+0x8b/0xa3
Sep 9 00:46:27 steel kernel: [vfs_ioctl+562/581] vfs_ioctl+0x232/0x245
Sep 9 00:46:27 steel kernel: [sys_ioctl+49/72] sys_ioctl+0x31/0x48
Sep 9 00:46:27 steel kernel: [sysenter_past_esp+95/133] sysenter_past_esp+0x5f/0x85
Sep 9 00:46:27 steel kernel: =======================
Sep 9 00:46:27 steel kernel: Code: a1 00 fe 3a c0 8d b8 84 03 00 00 8d 83 ec 00 00 00 8d 55 dc e8 91 f2 51 c6 64 a1 00 d0 3e c0 c7 00 01 00 00 00 8b 83 d4 00 00 00 <8b> 40 10 8b 80 ec 15 00 00 39 f0 73 9b a1 00 fe 3a c0 39 f8 79
Sep 9 00:46:27 steel kernel: EIP: [<f9c09bb2>] radeon_irq_wait+0xdf/0x138 [radeon] SS:ESP 0068:f4f92ed0
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2007-09-08 23:09 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-09-07 19:56 crash while playing bzflag Alex Riesen
2007-09-07 23:14 ` Chuck Ebbert
2007-09-08 18:11 ` Alex Riesen
2007-09-07 23:18 ` Michal Piotrowski
2007-09-08 18:12 ` Alex Riesen
2007-09-08 23:08 ` Alex Riesen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.