Re: DEVFS_FS

Re: DEVFS_FS

[Adam J. Richter]

>What is the approved substitute for DEVFS_FS that is marked
>obsolete?

	DEVFS_FS is not obsolete.  Rather, this is a case of misstatements
being repeated so many times that people start to believe it.

	1. Only devfs (or the lookup trapping facility used in my remake
of it) can configure new devices on demand based on accesses to files
missing from /dev, which allows for faster boot time (no need
to initialize device you won't use during this session), easier fixing
of device driver bugs, and potentially less memory utilization (the new
devfs implementation plus lookup traps have a code size of about 5kB combined;
the inode+dname utilization of devfs is less than sysfs due to there
being fewer nodes and you can run devfs without sysfs, and when
inode+dname memory utilization of both are fixed, the size of the
unnecessarily loaded kernel modules will dominate).

	2. Only devfs allows a device driver to pass suggested names
for arbitrary minor device numbers (e.g., /dev/sound/mixer,
/dev/sound/dsp).  Otherwise, a configuration file or a static /dev
has to know these things in advance.

	3. Only devfs allows a device driver to pass suggested names
for arbitrary related facilities that do not fall under the same
major device number (e.g., CD-ROM's, tapes).

	Fundamentally, the functionality of being able to trap accesses
to nonexistent devices and the ability to pass the additional string
information in #2 and #3 provide a more flexible user interface as
new kernel facilities are developed, and communicating the relevant
information through a virtual file system is useful for avoid
unnecessary serialization and deferring certain initializations (for
example, until later in a boot process).

	By the way, with devfs, especially under the new implementation,
it is not particularly difficult to remap the names using {,UN}REGISTER
events from a shadow file system (or directly from /dev if you also
want the default names), without the underlying configuration having
any prior knowledge about device numbers.  Many statements in Greg
Kroah-Hartmann's "udev and devfs - The final word" that seem to
contract this seem to me to be wrong or misleading when it would
be easy to be clearer (for example, "devfs does not handle the need
for dynamic major/minor numbers", when devfs does not care and not
know whether the major/minor device number passed to it were
dynamically allocated).

	This is not to say that I am "against udev" by every possible
interpretation of that term.  I think using sysfs or hotplug events to
create nodes in /dev (but usually not to load the corresponding kernel
modules) is a useful user interface improvement.  I also hope to see
the devfs interface evolve to do things like be able to statically export
the default /dev file mappers from kernel modules, just as hardware
device ID tables are exported, and, if sysfs memory consumption were
to improve radically (the sysfs backing store patch would really help),
then the memory usage, the _requirement_ of a user level helper
program (instead of it just being an option) and the loss of ability
to select devfs and sysfs independently _might_ be offset by
simplification of other user level helper code (by making /dev
maintenance one of multiple duties of a sysfs watching facility).

	I think that Linux users would be much better served by allowing
the devfs functionality to be actively maintained, improved and exported
to "non-devfs" systems (for example the tmpfs lookup trapping) in general,
and, in particular, by integrating my devfs rewrite instead of
refusing it (and I have offered to arrange it so that the kernel can
be configured with either the old or the new devfs implementation).
Whether this functionality is called "devfs" in the future is not
important, but providing its functionality for those who want it is,
and that is what is being undermined by not integrating the new devfs
implementation.

                    __     ______________ 
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: DEVFS_FS

[Bryan Henderson]

>DEVFS_FS is not obsolete.  Rather, this is a case of misstatements
>being repeated so many times that people start to believe it.

I thought it was ultimately a case of no one willing to maintain it.  What 
I read is that supporters of devfs got weary enough of the criticism that 
they finally packed up and went home.

To me, the criticism of devfs missed its most important feature:  It was 
fundamentally designed to eradicate device numbers altogether.  Device 
numbers are an ancient technology.  In today's world, a device's basic 
name should be something like /dev/sound/mixer instead of a small number.

If you miss that, and think devfs is about making mnemonic aliases for 
small numbers, then it's easy to see devfs as a poor approximation of 
udev.

>Only devfs (or the lookup trapping facility used in my remake
>of it) can configure new devices on demand based on accesses to files
>missing from /dev, which allows for faster boot time (no need
>to initialize device you won't use during this session), easier fixing
>of device driver bugs, and potentially less memory utilization

Why is that?  First, let me make sure I parsed the sentence right:  You're 
saying the only way to configure devices on demand such that you get all 
those benefits is by using your lookup trapping facility in conjunction 
with your version of devfs?  I don't see it.  Any device driver can put 
off configuring a device until it's accessed.  A loadable kernel module 
lets you put off driver loading and initialization until some time after 
boot.  Autoloading of LKMs lets you do that on demand.  And if autoloading 
isn't functional enough, your lookup trapping can do all manner of device 
initialization with or without devfs.  So where's the devfs connection?

Also, I'm not clear on what memory utilization or device initialization 
delays we're talking about.

Re: DEVFS_FS

[Jamie Lokier]

Bryan Henderson wrote:
> To me, the criticism of devfs missed its most important feature:  It was 
> fundamentally designed to eradicate device numbers altogether.  Device 
> numbers are an ancient technology.  In today's world, a device's basic 
> name should be something like /dev/sound/mixer instead of a small number.

I agree.

Numberless devices are so useful that even some non-devfs systems have
a few drivers which allocate a number and require userspace to do
horrible things like parse /proc/devices to open the device.

Versions of udev that use /lib/udev-state/devices.tar.bz2 to
workaround the limitation described below completely break this
numberless feature.

> If you miss that, and think devfs is about making mnemonic aliases for 
> small numbers, then it's easy to see devfs as a poor approximation of 
> udev.

In my experience, udev has not been as good as people say, because
while devfs just works, the udev-based system I'm using right now
requires all manner of manual modprobing before basic things like
pppd, openvpn and mtools work.

> >Only devfs (or the lookup trapping facility used in my remake
> >of it) can configure new devices on demand based on accesses to files
> >missing from /dev, which allows for faster boot time (no need
> >to initialize device you won't use during this session), easier fixing
> >of device driver bugs, and potentially less memory utilization
> 
> Why is that?  First, let me make sure I parsed the sentence right:  You're 
> saying the only way to configure devices on demand such that you get all 
> those benefits is by using your lookup trapping facility in conjunction 
> with your version of devfs?

Wrongly parsed.
He's saying _either_ devfs _or_ looking trapping is sufficient.

> Also, I'm not clear on what memory utilization or device initialization 
> delays we're talking about.

Examples: If you compile all USB drivers into the kernel, or if you
modprobe them during boot scripts, then boot time is extended by the
time taken to initialise the host controllers and connected devices.
If you compile in SCSI drivers, boot time is further extended by the
time taken to probe and initialise your SCSI devices.

The time for initialising devices which you don't want to use this
session can be quite significant, if you find a few seconds to be
significant.

Same goes for unnecessary memory utilisation.  Having drivers loaded
for devices which you aren't using can be significant, if you find a
few hundred kbytes significant.

-- Jamie

Re: DEVFS_FS

[Bryan Henderson]

>Examples: If you compile all USB drivers into the kernel, or if you
>modprobe them during boot scripts, then boot time is extended by the
>time taken to initialise the host controllers and connected devices.
>If you compile in SCSI drivers, boot time is further extended by the
>time taken to probe and initialise your SCSI devices.

>Having drivers loaded
>for devices which you aren't using can be significant,

So it sounds like you're talking about the feature of devfs in which the 
driver doesn't get loaded until someone refers to the /dev directory that 
refers to it.  But that's not a characteristic of devfs -- that's just 
devfs keeping up with the classic mknod system.  In the classic system (if 
you choose to set it up that way), the device driver doesn't get loaded 
until someone refers to its major number.  That's part of the general 
module autoloading function.

Fundamentally, I believe devfs has this problem worse than the mknod 
system.  Devfs strives to make meaningful names for devices, which means 
it has to some work, such as talk to the device, to create one.  With the 
mknod system, devices often have names that have nothing to do with the 
true identity of the device, so a name can simply be implied.  Taking SCSI 
as an example, a reasonable driver for a single parallel SCSI bus with one 
LU per target could simply assign minor number = target ID and do no 
initialization work at all.  But the same driver in devfs would want to 
talk to the 16 possible targets and make up names more meaningful than 
target ID for them -- and omit from the namespace the ones that don't 
answer.

Re: DEVFS_FS

[Adam J. Richter]

Bryan,

	I am writing a longer reply to your previous posting,
but I really must jump in and correct a couple of important
factual errors in your latest one.

On Fri, 12 Nov 2004 10:50:25 -0800, Bryan Henderson wrote:
>>Examples: If you compile all USB drivers into the kernel, or if you
>>modprobe them during boot scripts, then boot time is extended by the
>>time taken to initialise the host controllers and connected devices.
>>If you compile in SCSI drivers, boot time is further extended by the
>>time taken to probe and initialise your SCSI devices.

>>Having drivers loaded
>>for devices which you aren't using can be significant,

>So it sounds like you're talking about the feature of devfs in which the 
>driver doesn't get loaded until someone refers to the /dev directory that 
>refers to it.  But that's not a characteristic of devfs -- that's just 
>devfs keeping up with the classic mknod system.  In the classic system (if 
>you choose to set it up that way), the device driver doesn't get loaded 
>until someone refers to its major number.  That's part of the general 
>module autoloading function.

	Only with lookup trapping (either built into the old devfs
or as a separate facility), can you autoload on a reference to
/dev/cdroms/cdrom0, because you do not know the device number,
because, in turn, you do not know if it is scsi, iSCSI, IDE, or
proprietary dinosaur.  Also, in the case of IDE, knowing that
you are looking for the first IDE cdrom drive does still does
not tell you the major number (could be 3, 22, 33 or 34) or
the minor number (determined by which cable and drive number on
the cable).

>Fundamentally, I believe devfs has this problem worse than the mknod 
>system.  Devfs strives to make meaningful names for devices, which means 
>it has to some work, such as talk to the device, to create one.  With the 
>mknod system, devices often have names that have nothing to do with the 
>true identity of the device, so a name can simply be implied.  Taking SCSI 
>as an example, a reasonable driver for a single parallel SCSI bus with one 
>LU per target could simply assign minor number = target ID and do no 
>initialization work at all.  But the same driver in devfs would want to 
>talk to the 16 possible targets and make up names more meaningful than 
>target ID for them -- and omit from the namespace the ones that don't 
>answer.

	Your statement about SCSI and device numbers is factually
wrong, unless something has changed recently.  There are four
different major numbers for SCSI devices: cdrom (11 block),
tape (9 char), disk(8 block), and scsi generic (21 char).  None of
these, not even scsi generic, uses minor device numbers that
directly map to <channel,target,lun> ("channel" means "which
scsi cable").  Every(?) scsi controller driver calls scsi_scan_host
on initializing the controller card, and scsi_scan_host calls
scsi_probe_lun on every target (and if the target exists and has
meaningful other luns, then all of those luns as well), and the
current numbering scheme for the scsi device drivers needs this
information, since there is not predetermined mapping of
<channel,target,lun> to minor device number.

	Come to think of it, your example is actually an interesting
hypothetical example of why lookup trapping makes possible arrangements
that are not possible just by invoking modprobe on unused major device
numbers.  You see, under the scsi-3 set of standards, luns are not
limited to 3 bits; they're limited to 64 bits.  If memory serves, I
believe that Infiniband or iSCSI or both allow 64 bit luns, which
obviously will not fit in the current space given to minor device
numbers.  For this reason, I expect the high level scsi code will
never switch to using minor device numbers to directly encode
<channel,target,lun>.  On the other hand, some encoding of
<channel,target,lun> would do just fine as device names for
lookup trapping, which could invoke a handler that ask the SCSI
subsystem to scan just that <channel,target,lun> and create the
appropriate /dev entry.

	Okay, so your choice of scsi as an example was unlucky.
How about if we consider IDE instead?  The IDE drivers encode
the device into minor device numbers, and the same device
number is shared between CDROM and hard disk driver.  Let's
also ignore the complications that IDE block devices cover
four major block device numbers (3, 22, 33, and 34) and
ignore the fact that character device is 37 also maps to
these devices for IDE tape and that there is also IDE scsi.
Let's just talk about IDE cdrom versus disk selection for
block major 3.

	Under current devfs conventions, you do not have to an alias
like /dev/discs/disc1 that relies on sequential scanning until the
second hard disk is found.  Instead, you can use
/dev/ide/host0/bus0/target1/lun0 if you like.  The /dev/discs/disc1
symlink is not put in by the IDE host controller driver or the
generic IDE code.  The /dev/discs/disc1 symlink is created
when the ide-disk driver calls register_disk, which calls
devfs_add_disk.  There is no #ifdef DEVFS_FS code in the kernel
that I am aware of that forces additional sequential
scanning when devfs is enabled.

	Sequential type based names, what you kindly call
"meaningful names" (/dev/cdroms/cdrom0, /dev/discs/disc0,
/dev/tapes/tape1) can be implemented with or without devfs-style
lookup trapping.  Without lookup trapping, you just have to
identify all of the drives at boot and on hotplug events and
have some utility called by the boot and hotplug scripts make
the appropriate symbolic links.  The difference with lookup
trapping is that, as usual, it allows this to be deferred,
and also it allows the system to stop scanning once it has
found the requested device rather than having to scan all
devices.

	One minor caveat to this is that if you mix
names of the form /dev/ide/hostN/busN/targetN/lunN with
names of the form /dev/discs/discN, *and* you want to rely
on consistent numbering of /dev/discs/discN, then make
sure that any attempt to bind drivers on /dev/ide/hostN
will first make sure this has already been done for host0
through hostN-1.  Currently, I believe this is already done
for you in the case of parallel scsi and IDE, as I believe
they both scan all devices at initialization.  In any case,
it still potentially avoids unnecessary device scans in
comparison to not having lookup trapping.

	No implementation of devfs that I am aware of actually
_requires_ the kernel to do any additional device scanning
or requires user level programs to use the sequential type-based
device names for disks, tapes and CD/DVD drives.  In comparison,
lookup trapping could help the kernel _avoid_ some
unnecessary scanning for sites that want to use sequential
type-based names, because it does not require all of the devices
to be scanned in advance in order to be able to map a given
type+number name (e.g., "cdrom1" just requires scanning until
cdrom0 and cdrom1 are found).

                    __     ______________
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: DEVFS_FS

[Bryan Henderson]

>>So it sounds like you're talking about the feature of devfs in which the 

>>driver doesn't get loaded until someone refers to the /dev directory 
that 
>>refers to it.  But that's not a characteristic of devfs -- that's just 
>>devfs keeping up with the classic mknod system.  In the classic system 
(if 
>>you choose to set it up that way), the device driver doesn't get loaded 
>>until someone refers to its major number.  That's part of the general 
>>module autoloading function.
>
>                Only with lookup trapping (either built into the old 
devfs
>or as a separate facility), can you autoload on a reference to
>/dev/cdroms/cdrom0, because you do not know the device number,
>in turn, you do not know if it is scsi, iSCSI, IDE, or
>proprietary dinosaur. 

I don't get what example you're envisioning.  I do know that my cdrom 
device is scsi and I know (because I'm careful, of course) that its device 
number is (11,0).  So I can make a /dev/cdroms/cdrom0 such that the device 
drivers for my cdrom device get loaded not at boot time, but when I first 
access /dev/cdroms/cdrom0.  I assume you have an example in mind where, 
with classic mknod I would have to load the driver at boot time, but with 
devfs I could wait until the first access to /dev/cdroms/cdrom0. (Remember 
the assertion that's in question: that only devfs lets you defer occupying 
memory with a device driver until the device is used).

>>Taking SCSI 
>>as an example, a reasonable driver for a single parallel SCSI bus with 
one 
>>LU per target could simply assign minor number = target ID
>
>Your statement about SCSI and device numbers is factually
>wrong...  There are four different major numbers for SCSI devices:...

Well, not my statement but rather your interpretation of it.  When I said 
"a reasonable driver..." I didn't mean the one people use today, which 
doesn't do anything like what I described.  I was trying to make a simple 
hypothetical case to show the underlying concept that load-on-demand is 
harder in devfs than in classic mknod -- and furthermore that it's because 
devfs is expected to do better naming.  (Technically, you could set up 
devfs so that the device special file name is as meaningless as it is with 
classic mknod, and then it would be as easy to do autoload.  But that 
wouldn't really be in the spirit of devfs).

The fact that existing SCSI and ATA drivers don't make a reliable mapping 
between device number and useful device identity is a major selling point 
for devfs.  But to the extent that someone can live with the unreliable 
mapping (for example, I live with it without too much trouble with the 
cdrom device I described above), autoloading is simpler with classic mknod 
than with devfs.  And in this thread, I'm only arguing about whether devfs 
enables autoloading, not whether devfs enables good device naming.

Re: DEVFS_FS

[Adam J. Richter]

On Mon Nov 15 11:27:50 2004, Bryan Henderson wrote:

>>>So it sounds like you're talking about the feature of devfs in which the 

>>>driver doesn't get loaded until someone refers to the /dev directory 
>that 
>>>refers to it.  But that's not a characteristic of devfs -- that's just 
>>>devfs keeping up with the classic mknod system.  In the classic system 
>(if 
>>>you choose to set it up that way), the device driver doesn't get loaded 
>>>until someone refers to its major number.  That's part of the general 
>>>module autoloading function.
>>
>>                Only with lookup trapping (either built into the old 
>devfs
>>or as a separate facility), can you autoload on a reference to
>>/dev/cdroms/cdrom0, because you do not know the device number,
>>in turn, you do not know if it is scsi, iSCSI, IDE, or
>>proprietary dinosaur. 

>I don't get what example you're envisioning.  I do know that my cdrom 
>device is scsi and I know (because I'm careful, of course) that its device 
>number is (11,0).

	But you cannot in general.  For example, you cannot ship a
general Linux distribution this way, because you do not know that
the user's cdrom the scsi device.

>[...] (Remember 
>the assertion that's in question: that only devfs lets you defer occupying 
>memory with a device driver until the device is used).

	No, and I resent your attempt to try to change what I said
by removing qualifiers from it.  I was clearly talking about the
cases where the major and minor device numbers are not necessarily
known in advance.  In the future, if you want to try to quote my own
statements back to me, please use quotation.  I said:

|        Only with lookup trapping (either built into the old devfs
| or as a separate facility), can you autoload on a reference to
| /dev/cdroms/cdrom0, because you do not know the device number,
| because, in turn, you do not know if it is scsi, iSCSI, IDE, or
| proprietary dinosaur.

	This is a real advatnage of devfs-style lookup trapping.


>>>Taking SCSI 
>>>as an example, a reasonable driver for a single parallel SCSI bus with 
>one 
>>>LU per target could simply assign minor number = target ID
>>
>>Your statement about SCSI and device numbers is factually
>>wrong...  There are four different major numbers for SCSI devices:...

>Well, not my statement but rather your interpretation of it.  When I said 
>"a reasonable driver..." I didn't mean the one people use today, which 
>doesn't do anything like what I described.  I was trying to make a simple 
>hypothetical case to show the underlying concept that load-on-demand is 
>harder in devfs than in classic mknod --

	First of all, there are likely to be more ways to access
CD or DVD images in the future rather than fewer.  It may be more
common, for example, for people to want to access these drivers
over the net, or to have completely virtual drives in
the form of .iso files.  Requiring all of this to be done under
a single device number would just add extra cycles, increase
kernel code complexity, and further obscure potentially useful
hardware information from the user.  I am confident that the
lines of code involved in implementing this would exceed that
of my implementations of lookup traps plus devfs, and be harder
to compile out for users who don't want it.

	Secondly, you are now describing what I suspect would be
programmer-weeks if not months of development, integration,
documentation and other migration support, all for a system
that would make it harder for a user program to match devices
to hardware.  So, you are comparing apples to oranges if you want
to compare existing patches to a redefinition of major device numbers.

	In comparison, my smaller and more modular (not as in "kernel
module") devfs and lookup trapping mechanism makes it easier to do
further improvements.  So, it's likely that a similar effor to the
kind you describe could result in a much better system if one
started with devfs and lookup trapping (for example, one could
get modules to export their default /dev mappings statically, something
I was going to mention in that response to your first posting that I
should finally clean up and send).

>and furthermore that it's because devfs is expected to do better naming.

>(Technically, you could set up 
>devfs so that the device special file name is as meaningless as it is with 
>classic mknod, and then it would be as easy to do autoload.  But that 
>wouldn't really be in the spirit of devfs).

	"is expected to" and "spirit of devfs" are your own psychological
problems impeding your objectivity in technical evaluation.  A double
standard will not result in a clearer cost / benefit comparison.

	What it boils down to is this, you described some scenarios.
In the realistic interpretation of those scenarios, lookup trapping
can be somewhat helpful.  Interpreting those scenarios to be just the
narrow case of where it assumed that the first CD-ROM drive is a scsi
drive (an assumption most distrubtions cannot make) and where the
function of major device numbers has been substantially changed
would results in a special case where it might be worth the kernel
memory savings of not using compiling in lookup trapping if you're
not going to do anything else that would benefit from it.

	So, realistically, it seems to me that including lookup
trapping and devfs is potentially useful in common enough cases (for
example, general Linux distributions) that it should be worth
integrating into the kernel as an option.

                    __     ______________
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: DEVFS_FS

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Adam J. Richter wrote:
> On Mon Nov 15 11:27:50 2004, Bryan Henderson wrote:
> 
> 
>>>>So it sounds like you're talking about the feature of devfs in which the 
> 
> 
>>>>driver doesn't get loaded until someone refers to the /dev directory 
>>
>>that 
>>
>>>>refers to it.  But that's not a characteristic of devfs -- that's just 
>>>>devfs keeping up with the classic mknod system.  In the classic system 
>>
>>(if 
>>
>>>>you choose to set it up that way), the device driver doesn't get loaded 
>>>>until someone refers to its major number.  That's part of the general 
>>>>module autoloading function.
>>>
>>>               Only with lookup trapping (either built into the old 
>>
>>devfs
>>
>>>or as a separate facility), can you autoload on a reference to
>>>/dev/cdroms/cdrom0, because you do not know the device number,
>>>in turn, you do not know if it is scsi, iSCSI, IDE, or
>>>proprietary dinosaur. 
> 
> 
>>I don't get what example you're envisioning.  I do know that my cdrom 
>>device is scsi and I know (because I'm careful, of course) that its device 
>>number is (11,0).
> 
> 
> 	But you cannot in general.  For example, you cannot ship a
> general Linux distribution this way, because you do not know that
> the user's cdrom the scsi device.
> 
> 
>>[...] (Remember 
>>the assertion that's in question: that only devfs lets you defer occupying 
>>memory with a device driver until the device is used).
> 
> 
> 	No, and I resent your attempt to try to change what I said
> by removing qualifiers from it.  I was clearly talking about the
> cases where the major and minor device numbers are not necessarily
> known in advance.  In the future, if you want to try to quote my own
> statements back to me, please use quotation.  I said:
> 
> |        Only with lookup trapping (either built into the old devfs
> | or as a separate facility), can you autoload on a reference to
> | /dev/cdroms/cdrom0, because you do not know the device number,
> | because, in turn, you do not know if it is scsi, iSCSI, IDE, or
> | proprietary dinosaur.
> 
> 	This is a real advatnage of devfs-style lookup trapping.
> 

What is the advantage of autoloading on access to hardware backed device
files though?  As Bryan points out, the only apparent advantage is
saving a couple K in memory while the driver isn't loaded.

Claiming that trapping+autoloading will resolve device number issues is
a straw man.  You still have to walk the hardware, and apply policy for
which piece of hardware is 'cdrom0'.  The udev model does this very
effectively: where there is hardware backing the device files, there is
no good reason to not load the drivers and have policy set once and for
all on udevstart.

The only need for trapping that I can see is for modules that don't have
hardware backing them, such as tun and loop.

> 
> 
>>>>Taking SCSI 
>>>>as an example, a reasonable driver for a single parallel SCSI bus with 
>>
>>one 
>>
>>>>LU per target could simply assign minor number = target ID
>>>
>>>Your statement about SCSI and device numbers is factually
>>>wrong...  There are four different major numbers for SCSI devices:...
> 
> 
>>Well, not my statement but rather your interpretation of it.  When I said 
>>"a reasonable driver..." I didn't mean the one people use today, which 
>>doesn't do anything like what I described.  I was trying to make a simple 
>>hypothetical case to show the underlying concept that load-on-demand is 
>>harder in devfs than in classic mknod --
> 
> 
> 	First of all, there are likely to be more ways to access
> CD or DVD images in the future rather than fewer.  It may be more
> common, for example, for people to want to access these drivers
> over the net, or to have completely virtual drives in
> the form of .iso files.  Requiring all of this to be done under
> a single device number would just add extra cycles, increase
> kernel code complexity, and further obscure potentially useful
> hardware information from the user.  I am confident that the
> lines of code involved in implementing this would exceed that
> of my implementations of lookup traps plus devfs, and be harder
> to compile out for users who don't want it.
> 

Regardless of the 'four different major numbers for SCSI devices',
nobody claimed the solution was to have a unified major.

In fact, the opposite is true: userspace should not care at all about
major/minor.  Userspace should care about the semantics of the available
device nodes available in /dev.  With hardware, I want to _see_ the
device nodes created when I _add_ hardware.  udev handles this semantic
very well.  Where it fails is solely in the added devfs semantic of
modprobing on lookups of non-existing device nodes.  _This_ is the case
where some form of trapping would help very much.

For the time being though, I'm more than happy doing "echo tun >>
/etc/modules".

> 	Secondly, you are now describing what I suspect would be
> programmer-weeks if not months of development, integration,
> documentation and other migration support, all for a system
> that would make it harder for a user program to match devices
> to hardware.  So, you are comparing apples to oranges if you want
> to compare existing patches to a redefinition of major device numbers.
> 
> 	In comparison, my smaller and more modular (not as in "kernel
> module") devfs and lookup trapping mechanism makes it easier to do
> further improvements.  So, it's likely that a similar effor to the
> kind you describe could result in a much better system if one
> started with devfs and lookup trapping (for example, one could
> get modules to export their default /dev mappings statically, something
> I was going to mention in that response to your first posting that I
> should finally clean up and send).
> 
> 
>>and furthermore that it's because devfs is expected to do better naming.
> 
> 
>>(Technically, you could set up 
>>devfs so that the device special file name is as meaningless as it is with 
>>classic mknod, and then it would be as easy to do autoload.  But that 
>>wouldn't really be in the spirit of devfs).
> 
> 
> 	"is expected to" and "spirit of devfs" are your own psychological
> problems impeding your objectivity in technical evaluation.  A double
> standard will not result in a clearer cost / benefit comparison.
> 
> 	What it boils down to is this, you described some scenarios.
> In the realistic interpretation of those scenarios, lookup trapping
> can be somewhat helpful.  Interpreting those scenarios to be just the
> narrow case of where it assumed that the first CD-ROM drive is a scsi
> drive (an assumption most distrubtions cannot make) and where the
> function of major device numbers has been substantially changed
> would results in a special case where it might be worth the kernel
> memory savings of not using compiling in lookup trapping if you're
> not going to do anything else that would benefit from it.
> 
> 	So, realistically, it seems to me that including lookup
> trapping and devfs is potentially useful in common enough cases (for
> example, general Linux distributions) that it should be worth
> integrating into the kernel as an option.
> 
>                     __     ______________
> Adam J. Richter        \ /
> adam@yggdrasil.com      | g g d r a s i l
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBmlApdQs4kOxk3/MRAmkSAJwOoJYgWSIBzi6Gls4EwT/L/pRhfQCfWBGJ
tCIW8tmsYVQCkqtv4Kz/g6Y=
=s84Q
-----END PGP SIGNATURE-----

Re: DEVFS_FS

[Jamie Lokier]

Mike Waychison wrote:
> The udev model does this very
> effectively: where there is hardware backing the device files, there is
> no good reason to not load the drivers and have policy set once and for
> all on udevstart.

(a) boot time, (b) it's typically a few hundred k to be saved.

> The only need for trapping that I can see is for modules that don't have
> hardware backing them, such as tun and loop.

There's quite a few devices like that.

> In fact, the opposite is true: userspace should not care at all about
> major/minor.  Userspace should care about the semantics of the available
> device nodes available in /dev.  With hardware, I want to _see_ the
> device nodes created when I _add_ hardware.  udev handles this semantic
> very well.  Where it fails is solely in the added devfs semantic of
> modprobing on lookups of non-existing device nodes.  _This_ is the case
> where some form of trapping would help very much.

I agree, this is desirable behaviour.  But it isn't perfect.

Better than trapping would be if /dev/net/tun0 and so forth did
actually exist in /dev, to indicate the availability of those devices,
but the modules were still autoloaded on first access.

It would also be nicer if the device nodes were created to indicate
available/installed devices, without actually initialising the devices
or loading the bulk of their driver until they're needed.

In other words, if /dev were auto-generated based on all available
hardware and all available non-hardware drivers for the current
kernel, including modules not yet loaded.

That would be better IMNSHO, to the extent that it's possible.

Although it's necessary to load drivers for some devices (like ISA
cards) to detect them, it isn't necessary to load PCI drivers to
detect that they are present.  A mapping directly from detected PCI
ids to whatever udev needs could be extracted from the drivers at
compile time.  Such mappings nearly exist now, except that the mapping
is from ids to drivers, instead of ids to dev info.

-- Jamie

Re: DEVFS_FS

[Greg KH]

On Tue, Nov 16, 2004 at 07:26:47PM +0000, Jamie Lokier wrote:
> Mike Waychison wrote:
> > The only need for trapping that I can see is for modules that don't have
> > hardware backing them, such as tun and loop.
> 
> There's quite a few devices like that.

Do you have an exact list of such devices?  I would be interested in
seeing that if you did.

> Although it's necessary to load drivers for some devices (like ISA
> cards) to detect them, it isn't necessary to load PCI drivers to
> detect that they are present.  A mapping directly from detected PCI
> ids to whatever udev needs could be extracted from the drivers at
> compile time.  Such mappings nearly exist now, except that the mapping
> is from ids to drivers, instead of ids to dev info.

That's because we don't express that dev info within the drivers, as
it's usually not the drivers themselves that know or care about that dev
info.  It's the "class" cores that have that (input core, sound core,
usb core, etc.)  Drivers are just a conduit, and don't know, or care
about dev info at all.

So trying to make that mapping is pretty much impossible right now,
unfortunatly.

thanks,

greg k-h

Re: DEVFS_FS

[Bryan Henderson]

>>[...] (Remember 
>>the assertion that's in question: that only devfs lets you defer 
occupying 
>>memory with a device driver until the device is used).
>
>                No, and I resent your attempt to try to change what I 
said
>by removing qualifiers from it.  I was clearly talking about the
>cases where the major and minor device numbers are not necessarily
>known in advance.  In the future, if you want to try to quote my own
>statements back to me, please use quotation.

Rather than resent, you should try alternate interpretations of what I'm 
trying to do that might be less offsensive.  In this case I did not try to 
change what you said and didn't mean to attribute any statement you.  I 
tried to get you back on track with the specific thread of discussion, 
because it looks like you may be talking about something different than I 
am.  This specific subthread goes back to a question I asked when I didn't 
understand a statement in your original post about the virtues of devfs. 
Jamie tried to answer it, and I still didn't get it.  I pointed out what I 
thought was a flaw in Jamie's answer.  Your response to that post didn't 
seem to be germane to my criticism of Jamie's explanation.

All that aside, I'd still like to understand your original statement that 
(and I'm working from memory, so excuse me if it's not exact) only with 
devfs can you avoid boot time and memory associated with devices that you 
aren't using.  First, did you mean the old devfs as it stands has this 
property, or would some additional lookup trapping function have to be 
added?  Second, can I assume you're talking about delayed loading of the 
device driver module?  If those are true (which was Jamie's 
interpretation, at least), then can you give me an example of a scenario 
where devfs autoloading gives you this, but classic mknod autoloading 
doesn't?  You mentioned a case where you don't know whether a cdrom device 
is SCSI or something else.  I'll grant that autoload-on-major-number can't 
save you a boot-time module load there.  But how does devfs?

>>(Technically, you could set up 
>>devfs so that the device special file name is as meaningless as it is 
with 
>>classic mknod, and then it would be as easy to do autoload.  But that 
>>wouldn't really be in the spirit of devfs).
>
>"is expected to" and "spirit of devfs" are your own psychological
>problems impeding your objectivity in technical evaluation.  A double
>standard will not result in a clearer cost / benefit comparison.

Because you refer to it as a "problem," can I assume you disagree with my 
statement?  You think it would be in the spirit of devfs to program a 
driver to create a device file name as meaningless as what we've 
traditionally had in Linux (e.g. /dev/sdd meaning the 4th SCSI device the 
driver happened to come across), and that a user of devfs wouldn't expect 
any better?

We may have vastly different ideas about what devfs is good for.

Re: DEVFS_FS

[Mike Waychison]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jamie Lokier wrote:
> Mike Waychison wrote:
>>In fact, the opposite is true: userspace should not care at all about
>>major/minor.  Userspace should care about the semantics of the available
>>device nodes available in /dev.  With hardware, I want to _see_ the
>>device nodes created when I _add_ hardware.  udev handles this semantic
>>very well.  Where it fails is solely in the added devfs semantic of
>>modprobing on lookups of non-existing device nodes.  _This_ is the case
>>where some form of trapping would help very much.
> 
> 
> I agree, this is desirable behaviour.  But it isn't perfect.
> 
> Better than trapping would be if /dev/net/tun0 and so forth did
> actually exist in /dev, to indicate the availability of those devices,
> but the modules were still autoloaded on first access.

That would be cool. :)

> 
> It would also be nicer if the device nodes were created to indicate
> available/installed devices, without actually initialising the devices
> or loading the bulk of their driver until they're needed.
> 
> In other words, if /dev were auto-generated based on all available
> hardware and all available non-hardware drivers for the current
> kernel, including modules not yet loaded.
> 
> That would be better IMNSHO, to the extent that it's possible.
> 
> Although it's necessary to load drivers for some devices (like ISA
> cards) to detect them, it isn't necessary to load PCI drivers to
> detect that they are present.  A mapping directly from detected PCI
> ids to whatever udev needs could be extracted from the drivers at
> compile time.  Such mappings nearly exist now, except that the mapping
> is from ids to drivers, instead of ids to dev info.

The problem is that you do need to load the drivers for devices to see
any attached hardware.

For instance, consider the case where you boot of ide, but have an
attached scsi card.  There is no way for you to know what is behind that
scsi card until you load the drivers :(

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBmlmSdQs4kOxk3/MRAsXfAJ4iUpCaZxw/3AwvwU/177PzMgZrCwCgkP67
Q/Arb34MO3Z7t0p67hD9v4I=
=lxp3
-----END PGP SIGNATURE-----

Re: DEVFS_FS

[Bryan Henderson]

>In other words, if /dev were auto-generated based on all available
>hardware

Bear in mind that "all available hardware" may be a set completely 
impossible to ascertain.  Consider a computer attached to the Internet 
that uses ISCSI devices.  There would need to be a /dev entry for every 
ISCSI logical unit on the Internet.

I find that in discussing the various device naming schemes, people (and 
I'm not talking about present company) often have a tunnel vision view of 
a particular kind of device.  I don't think any one system yet invented 
works for everyone.

I find the strategies fall into two major categories:  1) addressing; and 
2) discovery.  With addressing, the user identifies the device in such a 
way that the system knows everything it needs to find it (or find that no 
such device exists).  Like identifying a web page by URL.  With discovery, 
the user identifies the device in a way that doesn't suggest how to reach 
it, like identifying a tape drive by the volume ID of the currently 
mounted tape.

Addressing says to me driver autoload on first access, lookup trapping, 
and no way to list the accessible devices. It's also great for devices 
that are purely conceptual -- they don't exist in any material form until 
they are used.  Discovery says to me hotplug events and, often, a 
configuration file (to describe the parameters of the discovery). 
Discovery is way more complicated (it adds state) and involves gratuitous 
access to devices, which is occasionally a problem.

Some of the strategies are hybrids.  For example, you address the device 
in enough detail that the system knows which driver to load, and then the 
driver (and/or user space program) does discovery to resolve the rest of 
the device name.

Looking at it historically:  when device numbers were invented, they were 
addresses.  (I mentioned in an earlier email a hypothetical SCSI device 
driver that assigns minor number = target ID -- that's how device numbers 
were originally supposed to be).  Ergo, /dev names were addresses.

Re: DEVFS_FS

[Jamie Lokier]

Bryan Henderson wrote:
> >In other words, if /dev were auto-generated based on all available
> >hardware
> 
> Bear in mind that "all available hardware" may be a set completely 
> impossible to ascertain.  Consider a computer attached to the Internet 
> that uses ISCSI devices.  There would need to be a /dev entry for every 
> ISCSI logical unit on the Internet.

I agree, that is neither feasible nor useful.
That's what I meant by the qualifier "to the extent that is possible".

We have the same problem with virtual filesystems like httpfs or
bittorrentfs: You can't list files in them, except for files the
system already knows about.

> I find the strategies fall into two major categories:  1) addressing; and 
> 2) discovery.  With addressing, the user identifies the device in such a 
> way that the system knows everything it needs to find it (or find that no 
> such device exists).  Like identifying a web page by URL.  With discovery, 
> the user identifies the device in a way that doesn't suggest how to reach 
> it, like identifying a tape drive by the volume ID of the currently 
> mounted tape.

That's a useful distinction to highlight.  There are devices that only
work nicely in one or other model.

-- Jamie