All of lore.kernel.org
 help / color / mirror / Atom feed
* snd-cmipci with Jack not working
@ 2003-01-29  9:27 Antti Boman
  2003-01-29 17:13 ` Takashi Iwai
  0 siblings, 1 reply; 9+ messages in thread
From: Antti Boman @ 2003-01-29  9:27 UTC (permalink / raw)
  To: alsa-devel

Hi,

I and a few other people have tried to get snd-cmipci (8738 integrated 
into ASUS A7V266 motherboard) work with Jack Audio Connection Kit, but 
with no success. We tried solving this problem in jack's developer list, 
but as there was no success I though (and was adviced) it might be the 
ALSA driver problem.

The thread about the problem can be found at
http://sourceforge.net/mailarchive/forum.php?thread_id=1580464&forum_id=3040

I've gathered the key information here:

- The sound when playing back is totally garbled, the pitch is lower
  than the original, and I get lots of clicking alongside. If I'm not
  using jack but alsa output, everything (playing and recording) is ok.

- I have the lowlat patch installed in kernel, but using or not using it
  doesn't seem to change anything.

- The irq for the "card" is 10, and doesn't seem to be shared. I also
  have .asoundrc set up.

- Jack server->client operation works ok, but playing back and recording
  cause the troubles.

- After watching /proc/asound/card0/pcm0p/sub0/status and
  /proc/asound/card0/pcm0c/sub0/status, it seems that the wake-up does
  not happen in time although using relatively large buffersizes, or:

- the playback and capture directions might be driven by different
  clocks (i.e. they run at different speeds)

- Using jackd with playback/capture only, it produces the following
  message:
   jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed

- If using kernel real-time patches with playback/capture only, the
  system locks up. This doesn't happen with both capture and playback
  enabled.

More can be found in the thread.

Anyway, we ran into dead-end. I'm very willing to answer any questions 
you may have, test any configuration you're suggesting, or debug 
anything you want me to debug.

And no, I still don't know if it's a Jack or ALSA driver problem. What 
I've read, though, is that people with other drivers (or chipsets) 
haven't had these kind of problems, and there are at least two reports 
on the same problem with users of C-Media 8738 with jack.

Thanks for any help!

-Antti Boman



-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-29  9:27 snd-cmipci with Jack not working Antti Boman
@ 2003-01-29 17:13 ` Takashi Iwai
  2003-01-29 23:48   ` Antti Boman
  2003-02-12 21:39   ` Antti Boman
  0 siblings, 2 replies; 9+ messages in thread
From: Takashi Iwai @ 2003-01-29 17:13 UTC (permalink / raw)
  To: Antti Boman; +Cc: alsa-devel

Hi,

At Wed, 29 Jan 2003 11:27:38 +0200,
Antti Boman wrote:
> 
> Hi,
> 
> I and a few other people have tried to get snd-cmipci (8738 integrated 
> into ASUS A7V266 motherboard) work with Jack Audio Connection Kit, but 
> with no success. We tried solving this problem in jack's developer list, 
> but as there was no success I though (and was adviced) it might be the 
> ALSA driver problem.
> 
> The thread about the problem can be found at
> http://sourceforge.net/mailarchive/forum.php?thread_id=1580464&forum_id=3040
 
at first, please check which chip model you are using and get the
register dump from /proc/asound/card0/cmipci.

> I've gathered the key information here:
> 
> - The sound when playing back is totally garbled, the pitch is lower
>   than the original, and I get lots of clicking alongside. If I'm not
>   using jack but alsa output, everything (playing and recording) is ok.
> 
> - I have the lowlat patch installed in kernel, but using or not using it
>   doesn't seem to change anything.
 
most likely the kernel scheduler is irrelevant.

> - The irq for the "card" is 10, and doesn't seem to be shared. I also
>   have .asoundrc set up.
> 
> - Jack server->client operation works ok, but playing back and recording
>   cause the troubles.
> 
> - After watching /proc/asound/card0/pcm0p/sub0/status and
>   /proc/asound/card0/pcm0c/sub0/status, it seems that the wake-up does
>   not happen in time although using relatively large buffersizes, or:
> 
> - the playback and capture directions might be driven by different
>   clocks (i.e. they run at different speeds)

hmm, it's difficult to know, but i don't think so.
could be an interrupt-related issue.

> - Using jackd with playback/capture only, it produces the following
>   message:
>    jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
 
that's weird.

> - If using kernel real-time patches with playback/capture only, the
>   system locks up. This doesn't happen with both capture and playback
>   enabled.
 
i can't produce this on my machine...


anyway, i updated the driver a bit.
please update the cvs version and report whether it works or not.


ciao,

Takashi


-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-29 17:13 ` Takashi Iwai
@ 2003-01-29 23:48   ` Antti Boman
  2003-01-30  9:44     ` Takashi Iwai
  2003-02-12 21:39   ` Antti Boman
  1 sibling, 1 reply; 9+ messages in thread
From: Antti Boman @ 2003-01-29 23:48 UTC (permalink / raw)
  To: Takashi Iwai; +Cc: alsa-devel

Thanks, Takashi! I'll get back to these Friday night, as I'm way too 
busy before that. There are some things included below:

Takashi Iwai wrote:
> at first, please check which chip model you are using and get the
> register dump from /proc/asound/card0/cmipci.

I thought I sent the chip model to the mailing list, but that was 
actually one of the off-list messages. The model seems to be "C-Media 
PCI CMI8738-MC6 (model 55)", as the dump of /proc/asound/card0/cmipci 
here tells:

C-Media PCI CMI8738-MC6 (model 55) at 0xd800, irq 10

00: 00 00 00 00
04: 04 00 00 00
08: 00 00 09 00
0c: 00 00 00 0a
10: 80 00 00 00
14: 00 00 00 00
18: 00 00 89 00
1c: ff ff ff ff
20: 10 40 80 00
24: 00 31 ff 00
28: ff ff ff ff
2c: ff ff ff ff
30: 00 00 00 00
34: 00 00 00 00
38: 00 00 00 00
3c: 00 00 00 00

>>I've gathered the key information here:
>>
>>- The sound when playing back is totally garbled, the pitch is lower
>>  than the original, and I get lots of clicking alongside. If I'm not
>>  using jack but alsa output, everything (playing and recording) is ok.
>>
>>- I have the lowlat patch installed in kernel, but using or not using it
>>  doesn't seem to change anything.
>  
> most likely the kernel scheduler is irrelevant.

The only effect it had was locking the whole machine when using 
playback/capture only.

>>- Using jackd with playback/capture only, it produces the following
>>  message:
>>   jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
>  
> that's weird.

Yes, and as you most probably know, it refers to alsa-lib/src/pcm/pcm.c

Oh, btw, I'm using alsa 0.9.0rc6, and as root (which you obviously 
already gathered from me using lowlatency patches).

Kernel 2.4.19 with AMD Athlon.

>>- If using kernel real-time patches with playback/capture only, the
>>  system locks up. This doesn't happen with both capture and playback
>>  enabled.
>  
> i can't produce this on my machine...

Here it does the same thing every time.

> anyway, i updated the driver a bit.
> please update the cvs version and report whether it works or not.

I'll do this on Friday, as I told. I'll let you (and the list) know 
immediately.

Thanks again for your help!

-Antti



-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-29 23:48   ` Antti Boman
@ 2003-01-30  9:44     ` Takashi Iwai
  2003-01-30 10:00       ` Antti Boman
  2003-01-30 10:07       ` Antti Boman
  0 siblings, 2 replies; 9+ messages in thread
From: Takashi Iwai @ 2003-01-30  9:44 UTC (permalink / raw)
  To: Antti Boman; +Cc: alsa-devel

At Thu, 30 Jan 2003 01:48:18 +0200,
Antti Boman wrote:
> 
> Thanks, Takashi! I'll get back to these Friday night, as I'm way too 
> busy before that. There are some things included below:
> 
> Takashi Iwai wrote:
> > at first, please check which chip model you are using and get the
> > register dump from /proc/asound/card0/cmipci.
> 
> I thought I sent the chip model to the mailing list, but that was 
> actually one of the off-list messages. The model seems to be "C-Media 
> PCI CMI8738-MC6 (model 55)", as the dump of /proc/asound/card0/cmipci 
> here tells:
 
ok, then the chip model is the same one as i have now.


> >>I've gathered the key information here:
> >>
> >>- The sound when playing back is totally garbled, the pitch is lower
> >>  than the original, and I get lots of clicking alongside. If I'm not
> >>  using jack but alsa output, everything (playing and recording) is ok.
> >>
> >>- I have the lowlat patch installed in kernel, but using or not using it
> >>  doesn't seem to change anything.
> >  
> > most likely the kernel scheduler is irrelevant.
> 
> The only effect it had was locking the whole machine when using 
> playback/capture only.

do you mean with LL-patch?  or with preempt patch?

> >>- Using jackd with playback/capture only, it produces the following
> >>  message:
> >>   jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
> >  
> > that's weird.
> 
> Yes, and as you most probably know, it refers to alsa-lib/src/pcm/pcm.c
 
if the above function is directly called from jack, then it should be
a bug of jack.  the error above says the passed argument is a null
pointer.


> Oh, btw, I'm using alsa 0.9.0rc6, and as root (which you obviously 
> already gathered from me using lowlatency patches).
> 
> Kernel 2.4.19 with AMD Athlon.
> 
> >>- If using kernel real-time patches with playback/capture only, the
> >>  system locks up. This doesn't happen with both capture and playback
> >>  enabled.
> >  
> > i can't produce this on my machine...
> 
> Here it does the same thing every time.

ah, sorry, what do you mean "kernel real-time patches"?
preempt-patch?

then i don't know.  i don't use them together with 2.4 kernels.


Takashi


-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-30  9:44     ` Takashi Iwai
@ 2003-01-30 10:00       ` Antti Boman
  2003-01-30 10:20         ` Takashi Iwai
  2003-01-30 10:07       ` Antti Boman
  1 sibling, 1 reply; 9+ messages in thread
From: Antti Boman @ 2003-01-30 10:00 UTC (permalink / raw)
  To: Takashi Iwai; +Cc: alsa-devel

Takashi Iwai wrote:
> At Thu, 30 Jan 2003 01:48:18 +0200,
> Antti Boman wrote:
>>>>
>>>>- I have the lowlat patch installed in kernel, but using or not using it
>>>> doesn't seem to change anything.
>>> 
>>>most likely the kernel scheduler is irrelevant.
>>
>>The only effect it had was locking the whole machine when using 
>>playback/capture only.
> 
> do you mean with LL-patch?  or with preempt patch?

Andrew Morton's 
(http://www.zip.com.au/~akpm/linux/schedlat.html#downloads) patch. Sorry 
about the flaws in terminology, I'm kind of half-dumb with these. If 
there are even difference with those ones, I have to check them out 
tonight and tell you then.

>>>>- Using jackd with playback/capture only, it produces the following
>>>> message:
>>>>  jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
>>> 
>>>that's weird.
>>
>>Yes, and as you most probably know, it refers to alsa-lib/src/pcm/pcm.c
>  
> if the above function is directly called from jack, then it should be
> a bug of jack.  the error above says the passed argument is a null
> pointer.

I know. But I don't know if it's directly called from jackd. I'll try 
and ask more on jackit-devel or try to find things myself. May be 
another dead-end, though.

> ah, sorry, what do you mean "kernel real-time patches"?
> preempt-patch?
> 
> then i don't know.  i don't use them together with 2.4 kernels.

Then again, even without enabling low latency (whatsoever) it doesn't 
work. I'm just afraid this won't be solved. And I'm not afraid for 
myself, just for the fact that 8738 is a highly used chip. Have you 
tried running jackd with the one of yours?

Cheers,
-a



-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-30  9:44     ` Takashi Iwai
  2003-01-30 10:00       ` Antti Boman
@ 2003-01-30 10:07       ` Antti Boman
  1 sibling, 0 replies; 9+ messages in thread
From: Antti Boman @ 2003-01-30 10:07 UTC (permalink / raw)
  To: Takashi Iwai; +Cc: alsa-devel

Confirming that it's Andrew Morton's low-latency patch.

-a



-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-30 10:00       ` Antti Boman
@ 2003-01-30 10:20         ` Takashi Iwai
  2003-01-30 12:29           ` Antti Boman
  0 siblings, 1 reply; 9+ messages in thread
From: Takashi Iwai @ 2003-01-30 10:20 UTC (permalink / raw)
  To: Antti Boman; +Cc: alsa-devel

At Thu, 30 Jan 2003 12:00:32 +0200,
Antti Boman wrote:
> 
> Takashi Iwai wrote:
> > At Thu, 30 Jan 2003 01:48:18 +0200,
> > Antti Boman wrote:
> >>>>
> >>>>- I have the lowlat patch installed in kernel, but using or not using it
> >>>> doesn't seem to change anything.
> >>> 
> >>>most likely the kernel scheduler is irrelevant.
> >>
> >>The only effect it had was locking the whole machine when using 
> >>playback/capture only.
> > 
> > do you mean with LL-patch?  or with preempt patch?
> 
> Andrew Morton's 
> (http://www.zip.com.au/~akpm/linux/schedlat.html#downloads) patch. Sorry 
> about the flaws in terminology, I'm kind of half-dumb with these. If 
> there are even difference with those ones, I have to check them out 
> tonight and tell you then.
 
basically LL-patch is safer...
anyway i recommed to use without LL-patch (or disabled via sysctl) as
the first tests.  otherwise we don't know which is the culprit.
as long as no heavy disk access, the normal scheduler should work
fine.


> >>>>- Using jackd with playback/capture only, it produces the following
> >>>> message:
> >>>>  jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
> >>> 
> >>>that's weird.
> >>
> >>Yes, and as you most probably know, it refers to alsa-lib/src/pcm/pcm.c
> >  
> > if the above function is directly called from jack, then it should be
> > a bug of jack.  the error above says the passed argument is a null
> > pointer.
> 
> I know. But I don't know if it's directly called from jackd. I'll try 
> and ask more on jackit-devel or try to find things myself. May be 
> another dead-end, though.
 
you can run non-stripped jackd in the source tree before installation,
and run from gdb to trace at which point this happens.


> > ah, sorry, what do you mean "kernel real-time patches"?
> > preempt-patch?
> > 
> > then i don't know.  i don't use them together with 2.4 kernels.
> 
> Then again, even without enabling low latency (whatsoever) it doesn't 
> work. I'm just afraid this won't be solved. And I'm not afraid for 
> myself, just for the fact that 8738 is a highly used chip. Have you 
> tried running jackd with the one of yours?

yes.  no hang up with -R and/or -P options, so far.

	# jackd -v -R -d alsa -d hw -r 44100 -p 512 -n 2

	# jackd -v -R -d alsa -d hw -r 44100 -p 512 -n 2 -P


Takashi


-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-30 10:20         ` Takashi Iwai
@ 2003-01-30 12:29           ` Antti Boman
  0 siblings, 0 replies; 9+ messages in thread
From: Antti Boman @ 2003-01-30 12:29 UTC (permalink / raw)
  To: Takashi Iwai; +Cc: alsa-devel

Takashi Iwai wrote:
> At Thu, 30 Jan 2003 12:00:32 +0200, Antti Boman wrote:
 >>
>>Andrew Morton's 
>>(http://www.zip.com.au/~akpm/linux/schedlat.html#downloads) patch. Sorry 
>>about the flaws in terminology, I'm kind of half-dumb with these. If 
>>there are even difference with those ones, I have to check them out 
>>tonight and tell you then.
>  
> basically LL-patch is safer...

Well, I've gone along with the recommendations of LAU Low-latency Howto 
at http://www.djcj.org/LAU/guide/Low_latency-Mini-HOWTO.php3 but then 
again, the basic problem I'm experiencing is not that enabling LL 
crashes the machine. It's the fact that I cannot get anything but 
garbled audio out of jackd with cmipci, along with the symptoms of 
playback/capture only.

But yes, I will get into those safety issues when I get the basics 
working :)

> anyway i recommed to use without LL-patch (or disabled via sysctl) as
> the first tests.

"echo 0 > /proc/sys/kernel/lowlatency" is enough, right?

> otherwise we don't know which is the culprit.
> as long as no heavy disk access, the normal scheduler should work
> fine.

Yes, I found that out, too. I just wanted to do the tests with both LL 
enabled and disabled, and obviously ran more tests with it enabled, as 
it caused more severe errors :)

>>>if the above function is directly called from jack, then it should be
>>>a bug of jack.  the error above says the passed argument is a null
>>>pointer.
>>
>>I know. But I don't know if it's directly called from jackd. I'll try 
>>and ask more on jackit-devel or try to find things myself. May be 
>>another dead-end, though.
>  
> you can run non-stripped jackd in the source tree before installation,
> and run from gdb to trace at which point this happens.

Ok, that's what I'll do tonight or tomorrow. I'll have all the weekend 
for solving this problem, and I will try to solve it even if it takes 
the whole weekend.

>>Then again, even without enabling low latency (whatsoever) it doesn't 
>>work. I'm just afraid this won't be solved. And I'm not afraid for 
>>myself, just for the fact that 8738 is a highly used chip. Have you 
>>tried running jackd with the one of yours?
> 
> yes.  no hang up with -R and/or -P options, so far.
> 	# jackd -v -R -d alsa -d hw -r 44100 -p 512 -n 2
> 	# jackd -v -R -d alsa -d hw -r 44100 -p 512 -n 2 -P

Ok. The second one definitely crashes every time.

Did you see the message in the the original thread in jackit-devel 
concerning watching /proc/asound/card0/pcm0p/sub0/status and 
/proc/asound/card0/pcm0c/sub0/status? If you did, was there anything 
abnormal there? If not, here's a link in case you want to see the results:

http://sourceforge.net/mailarchive/message.php?msg_id=3656330

Thanks a lot, Takashi, for helping me out. I'll do the tests and 
debugging you asked me to do as soon as I find the time for it. Tomorrow 
night if not before that.

-a



-------------------------------------------------------
This SF.NET email is sponsored by:
SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
http://www.vasoftware.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: snd-cmipci with Jack not working
  2003-01-29 17:13 ` Takashi Iwai
  2003-01-29 23:48   ` Antti Boman
@ 2003-02-12 21:39   ` Antti Boman
  1 sibling, 0 replies; 9+ messages in thread
From: Antti Boman @ 2003-02-12 21:39 UTC (permalink / raw)
  To: Takashi Iwai; +Cc: alsa-devel

Takashi Iwai wrote:
[snip]
 > anyway, i updated the driver a bit.
 > please update the cvs version and report whether it works or not.

Hi,

I tried RC7 without success. Does it have the change you mentioned in 
the driver, or should I go and try the CVS version? Well, I should, 
instead of asking this, but please drop a note if there's no use trying 
that.

I'll get into debugging jackd next.

Thanks a lot!

-a


The original message
--------------------
Takashi Iwai wrote:
> Hi,
> 
> At Wed, 29 Jan 2003 11:27:38 +0200,
> Antti Boman wrote:
> 
>>Hi,
>>
>>I and a few other people have tried to get snd-cmipci (8738 integrated 
>>into ASUS A7V266 motherboard) work with Jack Audio Connection Kit, but 
>>with no success. We tried solving this problem in jack's developer list, 
>>but as there was no success I though (and was adviced) it might be the 
>>ALSA driver problem.
>>
>>The thread about the problem can be found at
>>http://sourceforge.net/mailarchive/forum.php?thread_id=1580464&forum_id=3040
> 
>  
> at first, please check which chip model you are using and get the
> register dump from /proc/asound/card0/cmipci.
> 
> 
>>I've gathered the key information here:
>>
>>- The sound when playing back is totally garbled, the pitch is lower
>>  than the original, and I get lots of clicking alongside. If I'm not
>>  using jack but alsa output, everything (playing and recording) is ok.
>>
>>- I have the lowlat patch installed in kernel, but using or not using it
>>  doesn't seem to change anything.
> 
>  
> most likely the kernel scheduler is irrelevant.
> 
> 
>>- The irq for the "card" is 10, and doesn't seem to be shared. I also
>>  have .asoundrc set up.
>>
>>- Jack server->client operation works ok, but playing back and recording
>>  cause the troubles.
>>
>>- After watching /proc/asound/card0/pcm0p/sub0/status and
>>  /proc/asound/card0/pcm0c/sub0/status, it seems that the wake-up does
>>  not happen in time although using relatively large buffersizes, or:
>>
>>- the playback and capture directions might be driven by different
>>  clocks (i.e. they run at different speeds)
> 
> 
> hmm, it's difficult to know, but i don't think so.
> could be an interrupt-related issue.
> 
> 
>>- Using jackd with playback/capture only, it produces the following
>>  message:
>>   jackd: pcm.c:5851: snd_pcm_mmap_commit: Assertion 'pcm' failed
> 
>  
> that's weird.
> 
> 
>>- If using kernel real-time patches with playback/capture only, the
>>  system locks up. This doesn't happen with both capture and playback
>>  enabled.
> 
>  
> i can't produce this on my machine...
> 
> 
> anyway, i updated the driver a bit.
> please update the cvs version and report whether it works or not.
> 
> 
> ciao,
> 
> Takashi
> 
> 
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> http://www.vasoftware.com
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/alsa-devel
> 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2003-02-12 21:39 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-01-29  9:27 snd-cmipci with Jack not working Antti Boman
2003-01-29 17:13 ` Takashi Iwai
2003-01-29 23:48   ` Antti Boman
2003-01-30  9:44     ` Takashi Iwai
2003-01-30 10:00       ` Antti Boman
2003-01-30 10:20         ` Takashi Iwai
2003-01-30 12:29           ` Antti Boman
2003-01-30 10:07       ` Antti Boman
2003-02-12 21:39   ` Antti Boman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.