linux-hotplug.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* udev rules for new firewire driver stack
@ 2008-08-20 19:39 Stefan Richter
  2008-08-20 20:31 ` David Moore
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Stefan Richter @ 2008-08-20 19:39 UTC (permalink / raw)
  To: linux-hotplug

Hi all,

it appears that somebody added a new FireWire driver stack to Linux
kernel 2.6.22 but forgot to work with upstream udev and distributors
of udev to get suitable udev rules out to the public.

Now that endusers start to notice, I had a look at what might be
required.


1.)  SCSI devices run by firewire-sbp2 instead of sbp2

All is fine in this department as far as my testing shows.  (I use
Gentoo Linux.)  Inspection of current upstream udev shows two small
theoretical deficits:


1.a)  udev/extras/path_id/path_id

has got a handle_firewire() which is only called and executed correctly
if the sysfs device path matches */fw-host[0-9]*/*.  The new firewire
stack has different parent device names and a one step shorter device
hierarchy.  (There is no fw-host device anymore.)

Example /dev/disk/by-path link, ieee1394 stack:
pci-0000:05:04.0-ieee1394-0x00101003ae0001e0:00042c:0000 -> ../../sdd

Same device hooked to the same FireWire controller, firewire stack:
pci-fw4.0-scsi-0:0:0:0 -> ../../sdd

If you ask me, this doesn't matter.  In fact you could probably simply
remove handle_firewire() from extras/path_id/path_id because I have no
idea why anybody would be interested in the particular looks of the
by-path symlink.  After all, the path of hotpluggable devices changes
all the time.


1.b)  udev/extras/rule_generator/75-cd-aliases-generator.rules

There is

# the path of removable devices changes frequently
ACTION="add", SUBSYSTEM="block", SUBSYSTEMS="usb|ieee1394", ENV{ID_CDROM}="?*", ENV{GENERATED}!="?*", PROGRAM="write_cd_rules by-id", SYMLINK+="%c"

but I have not bothered to figure out how the CD/DVD-ROM/R/W aliases are
supposed to look like.  The 70-persistent-cd.rules which have been
collected on my PC over time are a confusing mess with useless
ENV{ID_PATH} items in it, for both ieee1394 and firewire driven devices.

More interesting would be aliases per ID_SERIAL, and that should be
obtained from ATTRS{ieee1394_id} which is provided by both sbp2 and
firewire-sbp2.  Right now, sbp2 uses a slightly different format in this
sysfs attribute per default, but I think I will send a kernel patch to
make the same format the default as in firewire-sbp2.


2.)  Networking

udev/extras/rule_generator/75-persistent-net-generator.rules contains a
reference to SUBSYSTEMS="ieee1394" in order to generate a comment.
I.e. a hypothetical IP-over-1394 driver for the firewire stack (not yet
written) won't get a nice comment over its persistent net rule.  This
can be amended when the driver is done.  I guess such a rule will turn
out to be something like
SUBSYSTEMS="firewire", ENV{COMMENT}="Firewire device $attr{guid})"


3.)  Most important:  Video and audio devices

Users should usually be able to access most if not all /dev/fw[0-9]*
character device files without having to be root.

These device files functionally replace the character device files
/dev/raw1394, /dev/video1394*, /dev/dv1394*.  But unlike the single
global /dev/raw1394 and the per-bus /dev/video1394* and per-bus
/dev/dv1394*, firewire-core's character device files are per node.
This offers the opportunity to have finer-grained permissions or
ownership or access control lists, based on device types.

For example, deny access per default, but allow access to a group or to
the owner of the console to character device files which belong to video
and audio devices.

Many if not all current libraw1394 based programs (and this are almost
all FireWire interfacing programs) also require access to the local node.
Alas firewire-core's sysfs interface does not provide any clue whether a
node is local or remote.  We can only treat all nodes which show
themselves as Linux nodes as if they were local nodes.

At least that's what I found out so far.  So here we go:


3.a)  udev/rules/gentoo/40-video.rules

contains

# IEEE1394 (firewire) devices
KERNEL="dv1394*|video1394*|raw1394*",	GROUP="video"

Here one could add

SUBSYSTEM="firewire", GROUP="video"

or change the existing rule to

KERNEL="dv1394*|video1394*|raw1394*|fw[0-9]*",	GROUP="video"

Alternatively, if you are keen on device type dependent permissions, use
something like this (in addition to the quoted ieee1394 rule):

--------------- 8< ---------------

# Set GROUP="video" for some IEEE 1394 device types, driven by the new firewire stack.
# We cannot use the GROUP directive because the significant device type attributes
# live in child devices. So change the group after the fact with chgrp.

# IIDC devices: industrial cameras and some webcams
SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x00010?",\
 PROGRAM="/bin/chgrp video /dev/%P"

# AV/C devices: camcorders, set-top boxes, TV sets, various audio devices, and more
SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x010001",\
 PROGRAM="/bin/chgrp video /dev/%P"

# libraw1394 clients typically also need access to the local node(s). Alas there is
# no simple way to tell local nodes apart from remote ones; here is a simple hack.
SUBSYSTEM="firewire", ATTR{vendor_name}="Linux Firewire", GROUP="video"

--------------- >8 ---------------

Note, libdc1394 based applications don't need the libraw1394 rule.  Nor
will hypothetical future programs which access /dev/fw* without help of
libraw1394 need it, unless they want to do nifty things on the local node.

Therefore the libraw1394 rule could perhaps be offloaded into an own rules
file, to be provided by libraw1394 packages.


3.b)  udev/rules/suse/40-suse.rules

Same as with Gentoo.


3.c)  Fedora

As far as I have heard, Fedora and probably RHEL don't set group
ownership but add an access control list entry, and that one is
dependent on the device type.  The console owner is granted access to
IIDC and AV/C devices that way.

The Fedora maintainers should also add access to local nodes, or they
need to modify (fix?) libraw1394 or libraw1394 client applications to
work without access to the local node.


3.d)  Others

I have no idea what other distros do with /dev/raw1394 and friends
and thus how their /dev/fw* rules should look like.


3.e)  Aliases for firewire character devices (symlinks)

Right now no application program is known to exist that would be able to
make use of aliases of (symlinks to) /dev/fw* character device files.
However, I spotted the following in udev/rules/redhat/40-redhat.rules:

KERNEL="fw*", PROGRAM="fw_unit_symlinks.sh %k %n", SYMLINK+="$result"

BTW, instead of the KERNEL match, I would recommend a SUBSYSTEM match.

A web search turned up that fw_unit_symlinks.sh looks or looked like
this:

--------------- 8< ---------------

#!/bin/sh

SYSFS=/sys
NAME=$1
NUM=$2

for i in "${SYSFS}${DEVPATH}/${NAME}.*/specifier_id"; do
    if [ ! -f "$i" ]; then
        continue
    fi
    SPECID="`cat $i`"
    if [ $((SPECID)) -eq $((0x00609e)) ]; then
        echo -n "sbp2-${NUM} "
    elif [ $((SPECID)) -eq $((0x00a02d)) ]; then
        SWVER="`cat ${i%/specifier_id}/version`"
        if [ $((SWVER)) -ge $((0x100)) -a $((SWVER)) -lt $((0x110)) ]; then
            echo -n "iidc${NUM} "
        elif [ $((SWVER)) -eq $((0x10001)) ]; then
            echo -n "avc${NUM} "
        elif [ $((SWVER)) -ge $((0x14000)) -a $((SWVER)) -le $((0x14001)) ]; then
            echo -n "vendorfw${NUM} "
        fi
    fi
done
echo

--------------- >8 ---------------

So this will create symlinks like iidc3, sbp2-7, avc5 end the likes.
These aren't persistent (only unique), because %n will change all the
time.  Since they provide symlinks which encode the types of available
units on the node, these symlinks may be of (some limited) use in the
future, provided that they would be available on all distributions.

However, since all existing applications retrieve the device type by
other means (typically per ioctl from firewire-core's config ROM cache
which the application or library then parses on its own) and since
applications need additional device type related information anyway, I
don't see a lot of benefit in these per-type symlinks.

Persistently named symlinks could be created too, for example by this
simple rule:

SUBSYSTEM="firewire", ATTR{guid}="0x*", SYMLINK+="firewire/$attr{guid}"

However, this too would provide only marginal benefit.  Users could
configure hypothetical future applications (which specialize on the new
firewire interface) to use always a desired
/dev/firewire/0x01234567deadbeef device rather than the ever changing
/dev/fw* devices.  However, a userfriendly application will be able to
figure out the right /dev/fw* device for a user-configured GUID on its
own without such a symlink.

So, all things considered, I think we don't need any of such symlinks at
all.


Thanks for your attention.  Suggestions and corrections welcome.
-- 
Stefan Richter
-===-=--- =--- =-=--
http://arcgraph.de/sr/

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
@ 2008-08-20 20:31 ` David Moore
  2008-08-20 20:57 ` Stefan Richter
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: David Moore @ 2008-08-20 20:31 UTC (permalink / raw)
  To: linux-hotplug

On Wed, 2008-08-20 at 21:39 +0200, Stefan Richter wrote:

> # Set GROUP="video" for some IEEE 1394 device types, driven by the new firewire stack.
> # We cannot use the GROUP directive because the significant device type attributes
> # live in child devices. So change the group after the fact with chgrp.
> 
> # IIDC devices: industrial cameras and some webcams
> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x00010?",\
>  PROGRAM="/bin/chgrp video /dev/%P"
> 
> # AV/C devices: camcorders, set-top boxes, TV sets, various audio devices, and more
> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x010001",\
>  PROGRAM="/bin/chgrp video /dev/%P"
> 

I tried playing with rules like this about a year ago, but it didn't
work because the spec_id was only available in the child nodes of the
firewire device, so udev couldn't match on it (see more below).  Has
that changed so these work now?

> 3.e)  Aliases for firewire character devices (symlinks)
> 
> Right now no application program is known to exist that would be able to
> make use of aliases of (symlinks to) /dev/fw* character device files.
> However, I spotted the following in udev/rules/redhat/40-redhat.rules:
> 
> KERNEL="fw*", PROGRAM="fw_unit_symlinks.sh %k %n", SYMLINK+="$result"
> 
> BTW, instead of the KERNEL match, I would recommend a SUBSYSTEM match.
> 
> A web search turned up that fw_unit_symlinks.sh looks or looked like
> this:
> 
.....

> So this will create symlinks like iidc3, sbp2-7, avc5 end the likes.
> These aren't persistent (only unique), because %n will change all the
> time.  Since they provide symlinks which encode the types of available
> units on the node, these symlinks may be of (some limited) use in the
> future, provided that they would be available on all distributions.

> So, all things considered, I think we don't need any of such symlinks at
> all.
> 

Yes, I think it was me who originally came up with this symlink script.
See here:

https://bugzilla.redhat.com/show_bug.cgi?id$0770

It was created because udev rules could not see the spec_id of the
devices and we wanted to selectively chgrp iidc and avc devices to the
console owner.  Thus, this script would create named symlinks and then
udev could simply chgrp based on the filename.

Now that ACLs are used in Fedora for dealing with console ownership,
this script and its udev rule are probably not very useful to anyone.

However, for other distributions that may not use ACLs, have you
confirmed that spec_id's are truly visible to udev rules now?

-David


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
  2008-08-20 20:31 ` David Moore
@ 2008-08-20 20:57 ` Stefan Richter
  2008-08-20 21:23 ` David Moore
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Stefan Richter @ 2008-08-20 20:57 UTC (permalink / raw)
  To: linux-hotplug

David Moore wrote:
> On Wed, 2008-08-20 at 21:39 +0200, Stefan Richter wrote:
> 
>> # Set GROUP="video" for some IEEE 1394 device types, driven by the new firewire stack.
>> # We cannot use the GROUP directive because the significant device type attributes
>> # live in child devices. So change the group after the fact with chgrp.
>>
>> # IIDC devices: industrial cameras and some webcams
>> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x00010?",\
>>  PROGRAM="/bin/chgrp video /dev/%P"
>>
>> # AV/C devices: camcorders, set-top boxes, TV sets, various audio devices, and more
>> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x010001",\
>>  PROGRAM="/bin/chgrp video /dev/%P"
>>
> 
> I tried playing with rules like this about a year ago, but it didn't
> work because the spec_id was only available in the child nodes of the
> firewire device, so udev couldn't match on it (see more below).  Has
> that changed so these work now?

This hasn't changed; hence I use chgrp on %P which is the parent.

I.e. these rules trigger on fwX.Y events but modify the (then already 
existing) /dev/fwX file.


>> 3.e)  Aliases for firewire character devices (symlinks)
>>
>> Right now no application program is known to exist that would be able to
>> make use of aliases of (symlinks to) /dev/fw* character device files.
>> However, I spotted the following in udev/rules/redhat/40-redhat.rules:
>>
>> KERNEL="fw*", PROGRAM="fw_unit_symlinks.sh %k %n", SYMLINK+="$result"
>>
>> BTW, instead of the KERNEL match, I would recommend a SUBSYSTEM match.
>>
>> A web search turned up that fw_unit_symlinks.sh looks or looked like
>> this:
>>
> .....
> 
>> So this will create symlinks like iidc3, sbp2-7, avc5 end the likes.
>> These aren't persistent (only unique), because %n will change all the
>> time.  Since they provide symlinks which encode the types of available
>> units on the node, these symlinks may be of (some limited) use in the
>> future, provided that they would be available on all distributions.
> 
>> So, all things considered, I think we don't need any of such symlinks at
>> all.
>>
> 
> Yes, I think it was me who originally came up with this symlink script.
> See here:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id$0770

Ah, interesting.

> It was created because udev rules could not see the spec_id of the
> devices and we wanted to selectively chgrp iidc and avc devices to the
> console owner.  Thus, this script would create named symlinks and then
> udev could simply chgrp based on the filename.
> 
> Now that ACLs are used in Fedora for dealing with console ownership,
> this script and its udev rule are probably not very useful to anyone.
> 
> However, for other distributions that may not use ACLs, have you
> confirmed that spec_id's are truly visible to udev rules now?

The rules mentioned above work for me (Gentoo, no ACLs).  The IIDC rule 
alone is enough for coriander.  The AV/C rule and the "Linux Firewire" 
rule get dvgrab et al going.

Instead of chgrp, these rules could also call setfacl or a setfacl wrapper.

However, I don't know whether the PROGRAM trick works for all 
distributions, even though it looks simple enough to me.
-- 
Stefan Richter
-===-=--- =--- =-=--
http://arcgraph.de/sr/

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
  2008-08-20 20:31 ` David Moore
  2008-08-20 20:57 ` Stefan Richter
@ 2008-08-20 21:23 ` David Moore
  2008-08-20 22:52 ` Stefan Richter
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: David Moore @ 2008-08-20 21:23 UTC (permalink / raw)
  To: linux-hotplug

On Wed, 2008-08-20 at 22:57 +0200, Stefan Richter wrote:
> David Moore wrote:
> > On Wed, 2008-08-20 at 21:39 +0200, Stefan Richter wrote:
> > 
> >> # Set GROUP="video" for some IEEE 1394 device types, driven by the new firewire stack.
> >> # We cannot use the GROUP directive because the significant device type attributes
> >> # live in child devices. So change the group after the fact with chgrp.
> >>
> >> # IIDC devices: industrial cameras and some webcams
> >> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x00010?",\
> >>  PROGRAM="/bin/chgrp video /dev/%P"
> >>
> >> # AV/C devices: camcorders, set-top boxes, TV sets, various audio devices, and more
> >> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x010001",\
> >>  PROGRAM="/bin/chgrp video /dev/%P"
> >>
> > 
> > I tried playing with rules like this about a year ago, but it didn't
> > work because the spec_id was only available in the child nodes of the
> > firewire device, so udev couldn't match on it (see more below).  Has
> > that changed so these work now?
> 
> This hasn't changed; hence I use chgrp on %P which is the parent.
> 
> I.e. these rules trigger on fwX.Y events but modify the (then already 
> existing) /dev/fwX file.
> 

Okay, I see.  The reason that didn't work on Fedora 7 was that a SYMLINK
rule can't use this %P trick since the target of the symlink can't be
overriden (at least to my knowledge).  And Fedora didn't like putting
chgrp commands directly in the udev rule since there was some PAM
console.perms magic elsewhere that dealt with the chgrping all in one
place based on filename.

Of course, for Fedora at least, all that is moot now since the PAM magic
is replaced with PolicyKit magic.  I'm not sure if other distributions
still use console.perms.

-David


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
                   ` (2 preceding siblings ...)
  2008-08-20 21:23 ` David Moore
@ 2008-08-20 22:52 ` Stefan Richter
  2008-08-21 18:32 ` Stefan Richter
  2008-08-21 18:57 ` David Moore
  5 siblings, 0 replies; 7+ messages in thread
From: Stefan Richter @ 2008-08-20 22:52 UTC (permalink / raw)
  To: linux-hotplug

David Moore wrote:
> On Wed, 2008-08-20 at 22:57 +0200, Stefan Richter wrote:
>> David Moore wrote:
>>> On Wed, 2008-08-20 at 21:39 +0200, Stefan Richter wrote:
>>>> SUBSYSTEM="firewire", ATTR{specifier_id}="0x00a02d", ATTR{version}="0x010001",\
>>>>  PROGRAM="/bin/chgrp video /dev/%P"
>>>>
>>> I tried playing with rules like this about a year ago, but it didn't
>>> work because the spec_id was only available in the child nodes of the
>>> firewire device, so udev couldn't match on it (see more below).  Has
>>> that changed so these work now?
>> This hasn't changed; hence I use chgrp on %P which is the parent.
>>
>> I.e. these rules trigger on fwX.Y events but modify the (then already 
>> existing) /dev/fwX file.
>>
> 
> Okay, I see.  The reason that didn't work on Fedora 7 was that a SYMLINK
> rule can't use this %P trick since the target of the symlink can't be
> overriden (at least to my knowledge).  And Fedora didn't like putting
> chgrp commands directly in the udev rule since there was some PAM
> console.perms magic elsewhere that dealt with the chgrping all in one
> place based on filename.
> 
> Of course, for Fedora at least, all that is moot now since the PAM magic
> is replaced with PolicyKit magic.  I'm not sure if other distributions
> still use console.perms.

The simplest alternative would of course just be

     SUBSYSTEM="firewire", GROUP="video"
or
     KERNEL="fw[0-9]*", GROUP="video"

or an equivalent PAM or CamelCaseKit based approach.  This would leave 
an opportunity unused to improve device security relative to raw1394, 
but that's not a big deal.

Another alternative is that we add a sysfs attribute to the parent 
device, e.g. one which contains a list of the specifier:version tuples 
of all unit directories of a node --- if that would make life easier for 
integrators.
-- 
Stefan Richter
-===-=--- =--- =-=-http://arcgraph.de/sr/

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
                   ` (3 preceding siblings ...)
  2008-08-20 22:52 ` Stefan Richter
@ 2008-08-21 18:32 ` Stefan Richter
  2008-08-21 18:57 ` David Moore
  5 siblings, 0 replies; 7+ messages in thread
From: Stefan Richter @ 2008-08-21 18:32 UTC (permalink / raw)
  To: linux-hotplug

David Moore wrote:
> On Wed, 2008-08-20 at 21:39 +0200, Stefan Richter wrote:
>> 3.e)  Aliases for firewire character devices (symlinks)
...
>> I spotted the following in udev/rules/redhat/40-redhat.rules:
>>
>> KERNEL="fw*", PROGRAM="fw_unit_symlinks.sh %k %n", SYMLINK+="$result"
...
>> this will create symlinks like iidc3, sbp2-7, avc5 end the likes.
>> These aren't persistent (only unique), because %n will change all the
>> time.  Since they provide symlinks which encode the types of available
>> units on the node, these symlinks may be of (some limited) use in the
>> future, provided that they would be available on all distributions.
> 
>> So, all things considered, I think we don't need any of such symlinks at
>> all.
>>
> 
> Yes, I think it was me who originally came up with this symlink script.
> See here:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id$0770
> 
> It was created because udev rules could not see the spec_id of the
> devices and we wanted to selectively chgrp iidc and avc devices to the
> console owner.  Thus, this script would create named symlinks and then
> udev could simply chgrp based on the filename.
> 
> Now that ACLs are used in Fedora for dealing with console ownership,
> this script and its udev rule are probably not very useful to anyone.

So, to summarize the need of symlink rules:

   - We don't need symlinks (aliases) for application support,
     neither type-oriented aliases, nor persistently named aliases.

   - Symlinks may be needed to fit the firewire device files into a
     distro's file permissions scheme though.  (This was actually the
     intent behind redhat/40-redhat.rules, which wasn't apparent to
     me when I read the rules files out of context.)

Ergo, there may be distro-specific symlinks but applications/libraries 
will not rely on any of those.  (Right?)
-- 
Stefan Richter
-===-=--- =--- =-=-http://arcgraph.de/sr/

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: udev rules for new firewire driver stack
  2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
                   ` (4 preceding siblings ...)
  2008-08-21 18:32 ` Stefan Richter
@ 2008-08-21 18:57 ` David Moore
  5 siblings, 0 replies; 7+ messages in thread
From: David Moore @ 2008-08-21 18:57 UTC (permalink / raw)
  To: linux-hotplug

On Thu, 2008-08-21 at 20:32 +0200, Stefan Richter wrote:
> David Moore wrote:
> > Yes, I think it was me who originally came up with this symlink script.
> > See here:
> > 
> > https://bugzilla.redhat.com/show_bug.cgi?id$0770
> > 
> So, to summarize the need of symlink rules:
> 
>    - We don't need symlinks (aliases) for application support,
>      neither type-oriented aliases, nor persistently named aliases.
> 
>    - Symlinks may be needed to fit the firewire device files into a
>      distro's file permissions scheme though.  (This was actually the
>      intent behind redhat/40-redhat.rules, which wasn't apparent to
>      me when I read the rules files out of context.)
> 
> Ergo, there may be distro-specific symlinks but applications/libraries 
> will not rely on any of those.  (Right?)

Yes, exactly right.

-David


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2008-08-21 18:57 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-08-20 19:39 udev rules for new firewire driver stack Stefan Richter
2008-08-20 20:31 ` David Moore
2008-08-20 20:57 ` Stefan Richter
2008-08-20 21:23 ` David Moore
2008-08-20 22:52 ` Stefan Richter
2008-08-21 18:32 ` Stefan Richter
2008-08-21 18:57 ` David Moore

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).