the userspace side of driverfs

the userspace side of driverfs

[Nicholas Miell]

I see documentation describing the kernel interface for driverfs, but
not much is available describing the userspace interface to driverfs --
i.e. the format of all those files that driverfs exports.

In order to prevent driverfs from becoming the maze of twisted files,
all different that is /proc, these details need to be specified now,
before it's too late.

Some issues I can think of off the top of my head:

- Can I safely assume that, for all normal files named X in driverfs,
that they have the exact same format and purpose?

- The "resource" files export resource structs, however the flags member
of the struct uses bits that aren't exported by the kernel and are
likely to change in the future. Also, some of the flags bits are
reserved for use by the bus that the resource lives on, but the bus type
isn't specified by the resource file, which requires the app to parse
the path name in order to figure out which bus the resource refers to.

- "name" isn't particularly consistent. Sometimes it requires parsing to
be useful ("PCI device 1234:1234", "USB device 1234:1234", etc.",
sometimes it's the actual device name, sometimes it's something strange
like "Hub/Port Status Changes".


- Nicholas

Re: the userspace side of driverfs

[Greg KH]

On Tue, Sep 10, 2002 at 06:18:37PM -0700, Nicholas Miell wrote:
> - "name" isn't particularly consistent. Sometimes it requires parsing to
> be useful ("PCI device 1234:1234", "USB device 1234:1234", etc.",
> sometimes it's the actual device name, sometimes it's something strange
> like "Hub/Port Status Changes".

The "Hub/Port..." thing will change, I have a large "struct
device_driver" patch for the USB code that is still being worked on
before going into the kernel, and this is changed in that patch..  For
all other "name" files, it's something that makes sense for the device,
and can be parsed by a human.  For some USB devices, they provide device
and manufacturer strings, so that information is provided.  Other
devices do not, so we guess the best that we can.

thanks,

greg k-h

Re: the userspace side of driverfs

[Patrick Mochel]

On 10 Sep 2002, Nicholas Miell wrote:

> I see documentation describing the kernel interface for driverfs, but
> not much is available describing the userspace interface to driverfs --
> i.e. the format of all those files that driverfs exports.
> 
> In order to prevent driverfs from becoming the maze of twisted files,
> all different that is /proc, these details need to be specified now,
> before it's too late.

I agree. There has been a lot of talk on this topic, but I don't think 
much has gotten down on paper, though there might be some in the 
archives...

The main ideal that we're shooting for is to have one ASCII value per
file. The ASCII part is mandatory, but there will definitely be exceptions
where we will have an array of or multiple values per file. We want to
minimize those instances, though. Both for the sake of easy parsing, but
also for easy formatting within the drivers.


> Some issues I can think of off the top of my head:
> 
> - Can I safely assume that, for all normal files named X in driverfs,
> that they have the exact same format and purpose?

Yes. One of the pipe dreams I have, which will hopefully become a reality 
in the future is to document the attributes when they're defined with a 
docbook-like comment. These can then be extracted and inserted into a 
database. 

We're working on a utility that abstracts the layout of driverfs from the
user. This will allow you do things like list all the devices and drivers
of a particular bus or class type, as well as display their attributes. 
With a database of attribute descriptions, you can provide desciptive 
context along with the value of the attribute.

> - The "resource" files export resource structs, however the flags member
> of the struct uses bits that aren't exported by the kernel and are
> likely to change in the future. Also, some of the flags bits are
> reserved for use by the bus that the resource lives on, but the bus type
> isn't specified by the resource file, which requires the app to parse
> the path name in order to figure out which bus the resource refers to.

The flags bit is a good point, and should probably be removed. 

Taking a step back, I would like to note that it would be nice to do 
something at a higher level with the resource structures; i.e. put them in 
struct device and deal with them in the driver model core. This is ways 
out, though if it happens, it will likely have repercussions in their 
driverfs representation..

> - "name" isn't particularly consistent. Sometimes it requires parsing to
> be useful ("PCI device 1234:1234", "USB device 1234:1234", etc.",
> sometimes it's the actual device name, sometimes it's something strange
> like "Hub/Port Status Changes".

'name' is better referred to as 'description'. It's a bus-specified string  
that describes the device. The bus is allowed to do as it pleases with it.

	-pat

Re: the userspace side of driverfs

[Nicholas Miell]

On Tue, 2002-09-10 at 21:38, Patrick Mochel wrote:
> 
> On 10 Sep 2002, Nicholas Miell wrote:
> 
> > - The "resource" files export resource structs, however the flags member
> > of the struct uses bits that aren't exported by the kernel and are
> > likely to change in the future. Also, some of the flags bits are
> > reserved for use by the bus that the resource lives on, but the bus type
> > isn't specified by the resource file, which requires the app to parse
> > the path name in order to figure out which bus the resource refers to.
> 
> The flags bit is a good point, and should probably be removed. 
> 

You need the flags, otherwise you can't distinguish dma/irq/mmio/ports.
The other flag bits are interesting, too.

 
- Nicholas

Re: the userspace side of driverfs

[Miquel van Smoorenburg]

In article <Pine.LNX.4.44.0209102122520.1057-100000@cherise.pdx.osdl.net>,
Patrick Mochel  <mochel@osdl.org> wrote:
>The main ideal that we're shooting for is to have one ASCII value per
>file. The ASCII part is mandatory, but there will definitely be exceptions
>where we will have an array of or multiple values per file. We want to
>minimize those instances, though. Both for the sake of easy parsing, but
>also for easy formatting within the drivers.

If you have multiple values per file, why not make it a directory with
multiple files in it, each with one value per file.

If you care about speed, perhaps you can provide a ".array" virtual
file in such a (or each) directory, that when read returns all files
in the directory in "filename: value" format so that you avoid the
while (readdir()) { open(); close(); } overhead.

This would be much cleaner, think for example of how you would
otherwise _write_ individual entries in such an array.

If you really want to get overboard, provide a sysctl() like function
that can read the entries in driverfs in binary. Like the existing
sysctl() in linux, but with an added TYPE_INT, TYPE_STRING etc flag
for each value for consistency. It too should be able to read an
entire directory as an array.

Then, convert procfs to the same interface ;)

Mike.

Re: the userspace side of driverfs

[Nicholas Miell]

On Wed, 2002-09-11 at 00:00, Miquel van Smoorenburg wrote:
> If you have multiple values per file, why not make it a directory with
> multiple files in it, each with one value per file.
> 

But subdirectories are child devices. Having array directories and
device directories would complicate the apps that have to parse this
data.
 
- Nicholas

Re: the userspace side of driverfs

[jw schultz]

On Tue, Sep 10, 2002 at 09:38:24PM -0700, Patrick Mochel wrote:
> I agree. There has been a lot of talk on this topic, but I don't think 
> much has gotten down on paper, though there might be some in the 
> archives...
> 
> The main ideal that we're shooting for is to have one ASCII value per
> file. The ASCII part is mandatory, but there will definitely be exceptions
> where we will have an array of or multiple values per file. We want to
> minimize those instances, though. Both for the sake of easy parsing, but
> also for easy formatting within the drivers.

Good so far.  When you have one value in a file the filename
tells you what it is.  What i don't want to see is more of
the multiple values in a file without labels or headings.
eg. /proc/sys/fs/inode-state (2.4.18):
	1792	133	0	0	0	0	0
I can't really trust documentation to keep up so the only
way i can be sure what these numbers are, is to look in the
kernel source.

Please, if you must have multiple values give them labels.

If it can only be 1 dimensional put one per line with a
label.  I don't care whether whether it 'label text:  value'
or 'label_text=value'  just as long as we are consistent
about the delimiters, capitalization (don't), whitespace and
underscore/dash.

If it needs to be 2 dimensional put the labels at the top as
a comment line (/^[;#]/d).  Using fixed width fields is asking
for trouble.  I prefer tab delimited but padding the fields
for alignment is OK as would be using tabs as long as we
agree on method and the labels don't have spaces.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: the userspace side of driverfs

[Matt Domsch]

> The main ideal that we're shooting for is to have one ASCII value per
> file. The ASCII part is mandatory, but there will definitely be exceptions
> where we will have an array of or multiple values per file. We want to
> minimize those instances, though. Both for the sake of easy parsing, but
> also for easy formatting within the drivers.

On IA-64, I've got the arch/ia64/kernel/efivars.c module that exports
/proc/efi/vars/{NVRAM-variables}.  It violates several rules of /proc
which I'd like to address in 2.5.x via driverfs.
1) It's in /proc but isn't process-related.
2) It exports its data as binary, not ascii.

Proc was chosen because it was simple, didn't require a major/minor
number, showed easily the set of NVRAM variables that were available
without needing a separate program to go and query a /dev/efivars file
to list them; cat and tar are sufficient for making copies of
variables and restoring them back again.  These exact features make
driverfs make sense too.

1) is easy to fix.  2) a little less so.  The data structure being
exported is a little over 2KB in length; The data is binary (itself a
variable length set of structures each with no ascii representation).
An ascii representation in "%02x" format will be longer than a 4K page
given to fill out and return.  Undoubtedly there's a better way to
handle this, and I'm open to suggestions.  The thing being exported is
efi_variable_t.

For such cases where the data being exported is really binary,
having a common set of parse/unparse routines would be nice. 

-Matt

-- 
Matt Domsch
Sr. Software Engineer, Lead Engineer, Architect
Dell Linux Solutions www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com
#1 US Linux Server provider for 2001 and Q1-2/2002! (IDC Aug 2002)

Re: the userspace side of driverfs

[jw schultz]

On Wed, Sep 11, 2002 at 11:13:17PM -0500, Matt Domsch wrote:
> > The main ideal that we're shooting for is to have one ASCII value per
> > file. The ASCII part is mandatory, but there will definitely be exceptions
> > where we will have an array of or multiple values per file. We want to
> > minimize those instances, though. Both for the sake of easy parsing, but
> > also for easy formatting within the drivers.
> 
> On IA-64, I've got the arch/ia64/kernel/efivars.c module that exports
> /proc/efi/vars/{NVRAM-variables}.  It violates several rules of /proc
> which I'd like to address in 2.5.x via driverfs.
> 1) It's in /proc but isn't process-related.
> 2) It exports its data as binary, not ascii.
> 
> Proc was chosen because it was simple, didn't require a major/minor
> number, showed easily the set of NVRAM variables that were available
> without needing a separate program to go and query a /dev/efivars file
> to list them; cat and tar are sufficient for making copies of
> variables and restoring them back again.  These exact features make
> driverfs make sense too.
> 
> 1) is easy to fix.  2) a little less so.  The data structure being
> exported is a little over 2KB in length; The data is binary (itself a
> variable length set of structures each with no ascii representation).
> An ascii representation in "%02x" format will be longer than a 4K page
> given to fill out and return.  Undoubtedly there's a better way to
> handle this, and I'm open to suggestions.  The thing being exported is
> efi_variable_t.
> 
> For such cases where the data being exported is really binary,
> having a common set of parse/unparse routines would be nice. 

I don't know what others think of this but i'd say that some
binary files are appropriate.  In a case like this i'd say
a files named 'nvram' and 'bios' or 'firmware' would be good
candidates for opaque binary structures and firmware.  This
is particularly the true if the data is purely related to
the device.  Ultimately it'd be nice to be able to upload
and download (install)  firmware this way.

Now if a datum is a parameter suitable for tuning i'd like
it made visible and updatable in an ASCII form.  In other
words i'd like to see an end to the proliferation of obscure
tools like hdparm.

These opinion does not necessarily reflect the views of
anyone else.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: the userspace side of driverfs

[Patrick Mochel]

> > For such cases where the data being exported is really binary,
> > having a common set of parse/unparse routines would be nice. 
> 
> I don't know what others think of this but i'd say that some
> binary files are appropriate.  

Not a chance. The values will be ASCII, and that's all there is to it. If 
I see someone exporting a binary file in driverfs, I'll submit a patch to 
remove it. :)

Matt, I'm interested in working on exporting the EFI variables in an 
ASCII manner, though time constraints are a bit stiff, and it's been a 
while sincce I looked at anything EFI. Stay tuned...

	-pat