Re: Hardware Abstraction Layer

[Greg KH <greg@kroah.com>]

On Thu, Sep 18, 2003 at 02:30:40PM -0400, Havoc Pennington wrote:
> On Thu, 2003-09-18 at 13:36, Greg KH wrote:
> > But my main point is, why store this info anywhere, when it is already
> > stored on your machine today? 
> 
> Keep in mind the original proposal at http://ometer.com/hardware.html
> which is that the HAL _merges_ information from various sources into a
> single list/tree of devices, and property set for each device.
> 
> So we may read some stuff from the USB hardware itself, then add some
> info provided by the kernel, then add some metadata from some systemwide
> files, then add some data that has been obtained by the desktop and
> stored per-user, then look at some blacklist, and finally we have a
> complete picture of everything known about that particular device.
> 
> But the exact set of things merged varies by device. The suggestion
> someone else posted about having "interfaces" is interesting in this
> respect.

Heh, watch out, that word is just too overloaded these days :)

For USB devices, it makes sense to do something like an "interface" as
USB devices can be compound devices with different drivers binding to
different "usb interfaces".  A USB speaker is the easiest example, the
usb audio driver binds to the audio portion of the device, and the hid
driver binds to the buttons part of the device.

Anyway, yes, you are right, if you want to merge from lots of different
sources, that is a good thing.

> One point of the HAL is to abstract where this information is coming
> from. Each info source is part of the "backend" not the
> application-visible API.

Ok, makes sense.

> >  And it is automatically already handled
> > properly today, without any need for "yet another abstraction layer" to
> > handle it?
> 
> Things aren't handled properly today in the sense that there's no UI on
> it, there's no integration with applications. To add that on the
> desktop/app level, we first have to have a single API that will work
> with different kinds of device and on different operating systems.

Hm, for a lot of this a UI isn't needed, right?  The user doesn't need
to know that a driver was just automatically loaded for their device,
correct?

Now I agree, that if something goes wrong (no driver found), a UI is a
very good thing to have.  I just don't want to see you all duplicate
work that has already been done :)

> One principle to keep in mind is: application code should not know the
> details of specific devices. If I add a new device model, apps should
> automatically use it. Ideally, I can even add a new kind of bus and apps
> can use devices on that bus; e.g. an app doesn't care if a scanner is
> SCSI or USB.

I completely agree.  That's one big reason why most USB devices use
existing kernel interfaces (SCSI, V4L, serial ports, etc.)

> From an OEM standpoint, if I submit a patch to one or a small
> well-defined set of projects, then my device should get used by all the
> interesting apps.
> 
> You may be lumping all userspace code together. Remember that the main
> desktop apps are built on dozens of libraries, all with opaque
> long-term-frozen ABIs that create additional abstraction layers.

I probably am, sorry.  Partly because I know what's going on down low in
the kernel, and don't pay attention to userspace much at all, "it's just
a test load for the kernel" is how most kernel developers feel :)

But remember, most OEMs do not write kernel drivers.  There are a few
exceptions, thankfully.  And that is what is required for almost all
devices to get working with linux today.

Again, there are very notable exceptions, the digital cameras supported
by gphoto2 and the USB scanners supported by xsane, both using userspace
only code.  I'm just pretty skeptical that something can be created that
would allow OEMs to plop into a userspace location that would enable
device support across multiple operating systems.  But please, prove me
wrong, I will be very happy.


> > Yes, I know this only works on Linux, and you want to be
> > "cross-platform".  Might I suggest you get the other OSs that you want
> > to support to also support what Linux already does, instead of trying to
> > duplicate work?
> 
> That doesn't help, unless the other OSs have _exactly_ the same
> filenames, interfaces, executables, and all other details. If there's
> _any_ difference between any interesting versions of Linux and any
> interesting versions of any other kernel, then we have to have the
> abstraction barrier. Apps can't contain #ifdef hell, it's not practical.
> App authors know _nothing_ about any specific hardware models or buses,
> and understand a single OS if that. They are on the level of "list the
> cameras, get the pictures from each"

I understand this.  The kernel doesn't contain #ifdef hell for that very
reason.  But this is my main point as to how this might get very tough
for having cross platform support.  The BSDs support lots of USB devices
today in a very different manner than Linux does.

> Interesting kernels at the moment seem to be Solaris, Linux, and *BSD
> including Darwin primarily. I don't see coordinating exact interfaces
> between those as practical.
> 
> If it is practical, it definitely involves writing a spec for what the
> interfaces are, and a conformance suite, and committing to maintaining
> spec conformance over time. I haven't seen anything like this for the
> Linux stuff yet. ;-)

I haven't either, but I would be interested in seeing such a thing.

> > Oh, what oses are you wanting to support?  I think FreeBSD already
> > handles much of what Linux does, with it's devfs implementation, but I
> > haven't looked at that in a long time to verify this.
> 
> Isn't devfs a device naming solution?

Yes it is.

> HAL really has little or nothing to do with device naming; from an app
> standpoint I don't think we care at all what's in /dev other than
> being able to find out what to use in there, see the "implementation
> goals" point #2 in http://ometer.com/hardware.html

Ok, David mentioned device naming, and I pointed out udev.  udev's goal
is to solve your point #2 for Linux.  Yes, it would be great to move
udev configuration into your HAL so that it can be configured in a
common manner.  I'll help out with this in any way possible.

> Think of it this way: it's a bug if we have a field in the UI like:
>  Device Node:   [   /dev/foo  ] 
> that the user has to fill in. Sometimes we might have to do it, but it's
> a UI bug for the OS/desktop combo as a whole system. Instead the UI
> should be:
>   Cameras:  [x] Minolta XYZ [ ] Canon
> where those may be on different buses/kernel subsystems.

That would be very nice.  Hopefully udev can help provide some of the
underlying glue that you need to determine this kind of information for
Linux.  It will contain a device database that you will be able to query
to get this kind of information.

> >  Now I agree that udev and
> > the hotplug scripts, and the raw hotplug events should get sent using
> > dbus events, but I was thinking that userspace programs (like GNOME and
> > KDE) that cared about those events would be listening for them.  I
> > didn't think there needed to be a whole intermediate layer for these
> > messages to have to go through.  But hey, I'm probably missing something
> > big here :)
> 
> Possibly you're expecting apps to understand the specifics of each
> kernel subsystem (USB, etc.) and have significant Linux-specific code.
> That's not feasible from the GNOME/KDE/OpenOffice.org/Mozilla/etc.
> standpoint.

Hm, yeah, sorry.  Again, I live too low in the OS stack :)

> To the user we display "here are your cameras"; and at some level there
> has to be an abstraction layer that puts things on the level of
> "cameras" not "USB devices" - that layer needs to be below the
> application, so that all apps have the same list of cameras.
> 
> (Cameras are just an example of course.)

And one of the most difficult examples, as previously mentioned (some
use gphoto2, some show up as scsi devices and must be mounted, some
others are not even supported at all...)

> > Why would loading a new driver really be of any interest to anyone?
> > Userspace can't do anything with that.  Userspace can only interact with
> > /dev nodes.
> 
> If you have a device and you don't know which driver to use, the desktop
> could ask the user which driver to use. Also, if I'm an OEM and I know
> my device works with a standard Linux driver, but that some deployed
> kernels won't know that, I could ideally ship the information in a
> little file of some kind for the user to install.

But if the driver was successfully loaded and is working, the user should
not care at all, right?

> Hopefully this mail helps clarify some of the goals. To use hardware
> effectively, apps need to have a hard dependency on an ABI for doing so.
> Hard deps must be portable and well-encapsulated/well-defined in terms
> of the semantics/ABI exported. Below this abstraction barrier, the
> various OS-specific stuff would be hooked in. So "HAL" essentially just
> means "the project that contains the abstraction barrier, and chains out
> to the per-OS per-device-model implementations" - my writeup has one
> speculation on what this project should look like, but I won't pretend
> to know for sure.

Yes, it does help a lot.

I'm happy to see this work happening, and if there's anything that you
need from the kernel, hotplug scripts, or udev that I can help out with,
please let me know.

thanks,

greg k-h


-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel