Automatic download and installation of drivers.

Automatic download and installation of drivers.

[Stamatis Mitrofanis]

Automatic download and installation of drivers. That is definately a 
good thing to have (in general). What we need now is to find:
- the best way to to use HTTP for the task.
- the best way to organize scripts/programs to have a general framework 
for requesting drivers of all kinds.
- actual database servers which are globally accessible to do the work.
- actual manufacturers to simply provide downloads in their ftp servers.

For HTTP there can be simple name=value requests that are passed to the 
server's CGI. The server redirects the client to where the actual file 
resides. The following holds:


wget http://www.kernel.org/cgi-bin/getdriver?bus=usb&pid=0x1234&vid=0x5678 <http://www.kernel.org/cgi-bin/getdriver?bus=usb&pid=0x1234&vid=0x5678>

Afterwards, the driver will get installed.

For the hotplug scripts I suggest that we create a new driver.agent 
which will process four actions (fetch/toss/install/uninstall) with 
appropriate parameters. It will be triggered by any other hotplug agent 
requiring a certain "resource" to satisfy an important request. The key 
thing here is that it should be acceptable to use the standard 
input/output of an agent.script when its called (like an ordinary 
program -- diagnostics should be sent to either stderr or syslog of 
course). This will be useful here since it's not very convenient to pass 
name=value pairs in the form of environment variables. Defining std 
input/output as "may be useful" for the hotplug scripts should be a good 
thing. install/uninstall get the DRIVER_RESOURCE parameter which is the 
local filename of the downloaded file. If a .tar.gz, they can guess the 
directory it unpacks to.

If anyone has any better ideas/suggestions, please let me know. I'm 
already working on the client-side for this (done install/uninstall for 
"targz.agent" driver subagent). It should be pretty easy to make the 
server-side DB and CGI linking to it.

Anyone to help?


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Tue, Oct 16, 2001 at 01:02:37AM +0100, Stamatis Mitrofanis wrote:
> Automatic download and installation of drivers. That is definately a 
> good thing to have (in general). 

Wow, the security implementations and other complexities involved here
are huge.  Let try a few examples:
	- installing a module requires root permissions.  You generally
	  do not want to run a compiler as root.
	- Running anything as root isn't a good idea, unless the program
	  has been audited for security problems.
	- Who is going to sign/verify the driver you just downloaded
	  to prevent a trojan from being installed?
	- Lots of user machines do not have compilers installed.  What
	  then?
	- Where are you going to find the .config file that the
	  currently running kernel was built against?  Without that, you
	  will not be able to successfully build that module.
	- What is the guarantee that older kernels will be able to load
	  newer modules?  Inter-kernel apis constantly change.  The odds
	  that this would work are quite slim.
	- Distros test their kernels as one package, drivers included.
	  Then they ship with those drivers.  Any new drivers fall under
	  the above problem.

And there are more problems that I can't think of right now.

But the main question I have is:
	What is the real problem that you are trying to solve?
and
	Why does the current kernel/driver situation not work for you?

thanks,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[David Brownell]

> What we need now is to find:
> - the best way to to use HTTP for the task.
> - the best way to organize scripts/programs to have a general framework 
>   for requesting drivers of all kinds.
> - actual database servers which are globally accessible to do the work.
> - actual manufacturers to simply provide downloads in their ftp servers.

Actually I think you've got the order backwards:  find systems that need
this kind of software distribution model first; then let their needs drive
development of such an alternative distribution scheme (and assurance
of its trustworthiness/security).  I can imagine local sites hosting some
servers for their local users, for example.

While I've sometimes been intrigued by such schemes, I'll have to be
contrarian in this case and say it's more interesting than useful at the
moment.  (Even for environments like Java, which was designed for
that model of software distribution.)  The current Linux model is that
driver distribution is tightly coupled to the associated kernel.  That is
known to work well.  What's the motivation for another model?

- Dave






_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Tuesday 16 October 2001 07:29, you wrote:
> > Automatic download and installation of drivers. That is definately a
> > good thing to have (in general).
> Wow, the security implementations and other complexities involved here
> are huge.  Let try a few examples:
> 	- installing a module requires root permissions.  You generally
> 	  do not want to run a compiler as root.
> 	- Running anything as root isn't a good idea, unless the program
> 	  has been audited for security problems.
> 	- Who is going to sign/verify the driver you just downloaded
> 	  to prevent a trojan from being installed?
> 	- Lots of user machines do not have compilers installed.  What
> 	  then?
> 	- Where are you going to find the .config file that the
> 	  currently running kernel was built against?  Without that, you

IMHO there are two choices for auto-installation. The easier one is to use 
distribution specific packages, then most of the problems do not apply. Only 
if you want to distribute drivers in source form you have to take care that:

- the sources of the installed kernel are available at a well-known place
- you have a well-defined compilation environment. You have to make rules 
which gcc version, which make version and so on has to be available and at 
which location (or alternatively you need a configuration file that describes 
their location). You need to be propared that the user has two compilers 
(gcc&kgcc).
- you need to know how to install and maybe also de-install modules in the 
system (it should provide a script for that)
- you need the configuration file of the installed kernel
- possibly you need the ability to re-compile the kernel itself if a needed 
configuration option is missing (for example enable V4L if the user installs 
a webcam). This can be quite complex which would be a reason to do this only 
with CML2. 
- ideally there would be a way for a installed driver package to add its own 
configuration options to the kernel. For example if you would install each 
drivers source in a directory /usr/src/linux/drivers/<driver-name>, then CML2 
could automagically include all /usr/src/linux/drivers/*/Config.in files.

This, of course, is a LOT of work.


> But the main question I have is:
> 	What is the real problem that you are trying to solve?
> and
> 	Why does the current kernel/driver situation not work for you?

1. It makes it much easier for a user to find a driver that's not included in 
his distribution. Linux drivers are usually named after the chipset's name or 
the first device that was supported. How can a user know that his Logitech 
Quickcam Pro 3000 needs the pwc (philips webcam) driver? And where can he 
find the driver?
2. (source driver packages only) right now the situation for drivers authors 
who want to distribute their drivers to end-users is really bad. Either they 
make it complicated for their users and distribute the drivers in source 
form. Even now I would guess the majority of Linux users is not able to 
compile their own drivers, and hopefully the percentage will be much lower in 
a couple of years.  Or the driver author provides binary packages for each 
version of each distribution. It is a lot of work and still won't help all 
users.  This is probably one of the reasons that so few hardware 
manufacturers provide Linux drivers.
 
bye...


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Keith Owens]

On Tue, 16 Oct 2001 09:58:38 +0200, 
Tim Jansen <tim@tjansen.de> wrote:
>2. (source driver packages only) right now the situation for drivers authors 
>who want to distribute their drivers to end-users is really bad.

<blatant plug>

Kernel build (kbuild) 2.5 is designed to make it easy to compile add on
code.  The user still needs the kernel source and the .config used to
build their kernel, plus their existing kernel objects.  The add on
source can be stored anywhere and will be compiled as if it were part
of the kernel source tree.

kbuild 2.5 gives full support for add on code, including make *config,
Makefile, kernel/module build, modules install, dependencies, the lot.
See http://sourceforge.net/projects/kbuild for the patch, read
Documentation/kbuild/kbuild-2.5.txt, in particular Shadow Trees.

</blatant plug>

Keith Owens, kbuild maintainer.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Tue, Oct 16, 2001 at 09:58:38AM +0200, Tim Jansen wrote:
> > But the main question I have is:
> > 	What is the real problem that you are trying to solve?
> > and
> > 	Why does the current kernel/driver situation not work for you?
> 
> 1. It makes it much easier for a user to find a driver that's not included in 
> his distribution. Linux drivers are usually named after the chipset's name or 
> the first device that was supported. How can a user know that his Logitech 
> Quickcam Pro 3000 needs the pwc (philips webcam) driver? And where can he 
> find the driver?

The current linux-hotplug interface solves this problem.  The driver
itself lets the kernel/world know for what devices it supports.  Coming
from the other direction (what driver supports this device) is just as
easy.  Look at modules.usbmap and modules.pcimap for examples.

> 2. (source driver packages only) right now the situation for drivers authors 
> who want to distribute their drivers to end-users is really bad. Either they 
> make it complicated for their users and distribute the drivers in source 
> form. Even now I would guess the majority of Linux users is not able to 
> compile their own drivers, and hopefully the percentage will be much lower in 
> a couple of years.  Or the driver author provides binary packages for each 
> version of each distribution. It is a lot of work and still won't help all 
> users.  This is probably one of the reasons that so few hardware 
> manufacturers provide Linux drivers.

The solution to this is to either live with the nightmare of supporting
binary packages for _every_ distro kernel release (1 kernel release from
RedHat is 11 kernels, multiply that by different versions of that distro
and the number of different distros, you have a huge mess) or simply put
the driver in the kernel source tree.

By putting the driver into the tree, all the problems go away, users
don't have to compile their own drivers, the drivers are all distributed
by the different Linux vendors, vendors don't have to update their
driver for every kernel api change, and everyone is happy.  Well, the
people who don't like open source drivers aren't happy, but we aren't
here to make their lives easier :)

thanks,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Tuesday 16 October 2001 18:52, Greg KH wrote:
> The current linux-hotplug interface solves this problem.  The driver
> itself lets the kernel/world know for what devices it supports.  Coming
> from the other direction (what driver supports this device) is just as
> easy.  Look at modules.usbmap and modules.pcimap for examples.

But how does this help for drivers that are not installed yet because they 
are not part of the kernel? This is the reason for downloading drivers.


> By putting the driver into the tree, all the problems go away, users
> don't have to compile their own drivers, the drivers are all distributed
> by the different Linux vendors, vendors don't have to update their
> driver for every kernel api change, and everyone is happy.  

I doubt that this will make hardware vendors happy:
- their devices cannot be supported before they are included in a kernel. 
Assuming a world where new devices are released with vendor-supported linux 
drivers this would mean that a new device cannot be released before their 
drivers are in the kernel and distributions picked this kernel up. 
- vendors lose control of their device drivers even though their quality is 
crucial for the success of their products

This is also a problem for users. Most distributions do update between 1 and 
4 times updates per year. This means that users have to wait for their driver 
up to one year before they can buy new devices or get an update for their 
driver. 

bye...





_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Tuesday 16 October 2001 09:35, you wrote:
> that model of software distribution.)  The current Linux model is that
> driver distribution is tightly coupled to the associated kernel.  That is
> known to work well.  What's the motivation for another model?

One motivation is to lower the barrier for hardware vendors who want to 
provide drivers. So far it is very difficult for them. Either they provide a 
dozen of RPM packages and still can not reach all Linux users (like nvidia) 
or they don't even try and support one or two distributions (AVM supports 
only SuSE on their ISDN adapter). The linux market is small enough, and by 
making it difficult to provide a single driver to all users you make is even 
less worthwhile. 

The advantage for users is that they can get drivers for new devices or 
updates without waiting for a new release of their distribution. It will also 
help users of, for example, debian and slackware who wouldn't get 
pre-packaged drivers otherwise, at least not before the driver is in the 
kernel. And those who would like to update their distribution, but the driver 
vendor hasn't released a newer package yet (SuSe 7.3 is out, but AVM only 
supports 7.1 and 7.2).

bye...

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Tue, Oct 16, 2001 at 08:44:04PM +0200, Tim Jansen wrote:
> On Tuesday 16 October 2001 18:52, Greg KH wrote:
> > The current linux-hotplug interface solves this problem.  The driver
> > itself lets the kernel/world know for what devices it supports.  Coming
> > from the other direction (what driver supports this device) is just as
> > easy.  Look at modules.usbmap and modules.pcimap for examples.
> 
> But how does this help for drivers that are not installed yet because they 
> are not part of the kernel? This is the reason for downloading drivers.

If the running (old) kernel does not know about the devices, but a
future kernel does, then you have all the same problems that I listed
before (kernel api, compilers, .config, etc.)  Download the whole
kernel, it's the only way, as there is not going to be a common binary
or source api for Linux kernel drivers, like other operating systems
have.

> > By putting the driver into the tree, all the problems go away, users
> > don't have to compile their own drivers, the drivers are all distributed
> > by the different Linux vendors, vendors don't have to update their
> > driver for every kernel api change, and everyone is happy.  
> 
> I doubt that this will make hardware vendors happy:
> - their devices cannot be supported before they are included in a kernel. 
> Assuming a world where new devices are released with vendor-supported linux 
> drivers this would mean that a new device cannot be released before their 
> drivers are in the kernel and distributions picked this kernel up. 

Vendors who work with open source drivers, typically have their Linux
drivers updated _before_ the hardware hits the shelves.  See the keyspan
and edgeport USB drivers for examples of this.

> - vendors lose control of their device drivers even though their quality is 
> crucial for the success of their products

Then the vendors have the maintainer of the driver be an employee.  This
is quite common (myself included.)  Look at most all of the SCSI drivers
for other examples.

> This is also a problem for users. Most distributions do update between 1 and 
> 4 times updates per year. This means that users have to wait for their driver 
> up to one year before they can buy new devices or get an update for their 
> driver. 

Then if a vendor wants a specific driver built for a specific distro's
kernel, due to the distro not having support for it, they build it.
I've done this in the past for some companies, it's not that hard, just
a pain getting all of the .config settings correct (which is what any
"automated" system would have to do also.)

thanks,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[David Brownell]

> One motivation is to lower the barrier for hardware vendors who want to 
> provide drivers. So far it is very difficult for them. Either they provide a 
> dozen of RPM packages and still can not reach all Linux users (like nvidia) 
> or they don't even try and support one or two distributions (AVM supports 
> only SuSE on their ISDN adapter). 

Easy fix:  provide a patch to the kernel source, rather than going with
a binary distribution model.  The model of binary driver distributions
is a problem.  It's my understanding that most hardware vendors would
rather not be in that game, but MSFT forced them to play that way.


> The advantage for users is that they can get drivers for new devices or 
> updates without waiting for a new release of their distribution. 

With a patch, "knowledgeable" users can get going right away.  All
others are going to need to wait for a new release in any case, or
will ask their "knowledgeable user" friends for help.

- Dave



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[David Brownell]

Yep, a better way to manage source code is the Linux Way!  :)

Of course, it's still tricky to answer the "is there a Linux driver
for this hardware".  More vendors should have those answers
available on their websites.

- Dave


----- Original Message ----- 
From: "Keith Owens" <kaos@ocs.com.au>
To: <linux-hotplug-devel@lists.sourceforge.net>
Sent: Tuesday, October 16, 2001 4:38 AM
Subject: Re: Automatic download and installation of drivers.


> On Tue, 16 Oct 2001 09:58:38 +0200, 
> Tim Jansen <tim@tjansen.de> wrote:
> >2. (source driver packages only) right now the situation for drivers authors 
> >who want to distribute their drivers to end-users is really bad.
> 
> <blatant plug>
> 
> Kernel build (kbuild) 2.5 is designed to make it easy to compile add on
> code.  The user still needs the kernel source and the .config used to
> build their kernel, plus their existing kernel objects.  The add on
> source can be stored anywhere and will be compiled as if it were part
> of the kernel source tree.
> 
> kbuild 2.5 gives full support for add on code, including make *config,
> Makefile, kernel/module build, modules install, dependencies, the lot.
> See http://sourceforge.net/projects/kbuild for the patch, read
> Documentation/kbuild/kbuild-2.5.txt, in particular Shadow Trees.
> 
> </blatant plug>
> 
> Keith Owens, kbuild maintainer.
> 
> 
> _______________________________________________
> Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
> Linux-hotplug-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Wednesday 17 October 2001 04:01, David Brownell wrote:
> Easy fix:  provide a patch to the kernel source, rather than going with
> a binary distribution model.  The model of binary driver distributions
> is a problem. 
> With a patch, "knowledgeable" users can get going right away.  All
> others are going to need to wait for a new release in any case

But this means, in other words, that Linux is not competitive in this area. 
Then regular users have to wait up to 6 months before they can use new 
devices because they have to wait for the next Red Hat distribution while Mac 
and Windows users can already use the device. It would also force them to 
update a stable system only for getting a new device driver.
And it would result in hardware manufactures being unable to sell their new 
devices to the majority of Linux users before most distributions have been 
updated.

bye...






_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[David Brownell]

The _full_ quote of that last paragraph contains the disproof to
your claim about timeliness and competitiveness ... 

> With a patch, "knowledgeable" users can get going right away.  All
> others are going to need to wait for a new release in any case, or
> will ask their "knowledgeable user" friends for help.

It's well known that even in the Windows world, most support
comes from friendly local experts ("power users").  Those are
the folk that get called in when a vendor's driver disk doesn't
install correctly (the ones I get usually don't).

So far as I know, there isn't even an authoritative list of all
the USB and CardBus devices supported by, say, the 2.4.12
kernel.  Or even a good way to extract the subset of that list
that's packaged as the MODULE_DEVICE_TABLE entries,
short of building a kernel with every driver configured as a
module (a bit painful IMO).

- Dave


----- Original Message ----- 
From: "Tim Jansen" <tim@tjansen.de>
To: "David Brownell" <david-b@pacbell.net>; <linux-hotplug-devel@lists.sourceforge.net>
Sent: Wednesday, October 17, 2001 11:47 AM
Subject: Re: Automatic download and installation of drivers.


> On Wednesday 17 October 2001 04:01, David Brownell wrote:
> > Easy fix:  provide a patch to the kernel source, rather than going with
> > a binary distribution model.  The model of binary driver distributions
> > is a problem. 
> > With a patch, "knowledgeable" users can get going right away.  All
> > others are going to need to wait for a new release in any case
> 
> But this means, in other words, that Linux is not competitive in this area. 
> Then regular users have to wait up to 6 months before they can use new 
> devices because they have to wait for the next Red Hat distribution while Mac 
> and Windows users can already use the device. It would also force them to 
> update a stable system only for getting a new device driver.
> And it would result in hardware manufactures being unable to sell their new 
> devices to the majority of Linux users before most distributions have been 
> updated.
> 
> bye...
> 
> 
> 



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Wednesday 17 October 2001 21:24, David Brownell wrote:
> > With a patch, "knowledgeable" users can get going right away.  All
> > others are going to need to wait for a new release in any case, or
> > will ask their "knowledgeable user" friends for help.
> It's well known that even in the Windows world, most support
> comes from friendly local experts ("power users").  Those are
> the folk that get called in when a vendor's driver disk doesn't
> install correctly (the ones I get usually don't).

But this is one of the problems with Linux and newbies - even with today's 
distributions there are far too many things you need "knowledgable" users 
for. This is definitely not the way it should be, and Windows shows that it 
is not neccessary. While it is true that it the driver installation fails 
sometimes you can be sure that this will improve over the next years. 
Actually the frequent failures of driver installations are probably the main 
reasons that Microsoft started their driver certification program.


> So far as I know, there isn't even an authoritative list of all
> the USB and CardBus devices supported by, say, the 2.4.12
> kernel. 

You wouldn't need this in the kernel for automatic downloads. Actually the 
idea of automatic downloadable drivers is to provide drivers that are NOT in 
the kernel. If the user connects a new device and the system doesn't find a 
driver for it it would request one from some server. This server could either 
provide a driver itself, based the device's vendor and model id, or forward 
the system to the vendor's server which would then try to find one for the 
model.

bye...



_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Thu, Oct 18, 2001 at 01:31:17AM +0200, Tim Jansen wrote:
> 
> You wouldn't need this in the kernel for automatic downloads. Actually the 
> idea of automatic downloadable drivers is to provide drivers that are NOT in 
> the kernel. If the user connects a new device and the system doesn't find a 
> driver for it it would request one from some server. This server could either 
> provide a driver itself, based the device's vendor and model id, or forward 
> the system to the vendor's server which would then try to find one for the 
> model.

But how do you resolve all of the previous points I raised about
compilers, .config files, in kernel apis, and security with regards to
"automatic downloads"?

thanks,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Dmitri]

[-- Attachment #1: Type: text/plain, Size: 2879 bytes --]

Quoting Greg KH <greg@kroah.com>:

> On Thu, Oct 18, 2001 at 01:31:17AM +0200, Tim Jansen wrote:
> > 
> > You wouldn't need this in the kernel for automatic downloads. Actually the 
> > idea of automatic downloadable drivers is to provide drivers that are NOT in 
> > the kernel. If the user connects a new device and the system doesn't find a 
> > driver for it it would request one from some server. This server could either 
> > provide a driver itself, based the device's vendor and model id, or forward 
> > the system to the vendor's server which would then try to find one for the 
> > model.
> 
> But how do you resolve all of the previous points I raised about
> compilers,

Just yesterday I had to replace gcc 3.0.1 with old but working
egcs-2.91.66. The C download was just 1.5 MBytes, and C++ was
only 1.5 MBytes more - though you wouldn't need C++ for the kernel.

This means that gcc is just another dependency that the script will
resolve before even trying to fetch and compile the driver. The
size of the required download is acceptable even for modem users,
and that has to be done only once.

Of course, distributions can make the gcc (suitable for kernel
compilations) a required (base) package.

> .config files,

I think they are available in "kernel-devel" package or something
like that. They easily can be shipped as part of the kernel RPM.
The kernel-devel package is already available in RH, for example.

Of course, the kernel itself can contain a copy of its own .config
file and export it through /proc fs, along with all compiler options,
that would be neat :-) and only requires an optional module.

> in kernel apis,

That's not something new. APIs change all the time, everywhere, not
only in OSes. Programmers just need to use #if ... #endif and maybe
the driver should be marked as "suitable" or "tested" on some range
of kernel releases.

> and security with regards to "automatic downloads"?

Linus already signs his kernel releases. There is no reason why
driver maintainers can't join the web of trust. Then GPG will
cheerfully accept signatures of partially trusted signers. It is
very easy to do, publishers only need to get their public key
signed by many other publishers, and the threshold for the sufficient
trust can be hardcoded to be fairly high.

/***/

It is clear to me that Linux needs a mechanism for fetching and
installing 3rd party drivers. The proposed technology seems to
be quite simple, and requires very little development because
all components are already here. I think, a single script can
definitely accomplish searching, downloading, compiling (kbuild?)
and installing the driver. This project is all about organizing
software and people to do it right.

Thanks,
Dmitri

-- 
panic("esp: penguin status phase.");
(Panic message in the kernel.)

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Automatic download and installation of drivers.

[Stamatis Mitrofanis]

Dmitri wrote:

>Quoting Greg KH :
>
>>On Thu, Oct 18, 2001 at 01:31:17AM +0200, Tim Jansen wrote:
>>
>>>You wouldn't need this in the kernel for automatic downloads. Actually the 
>>>idea of automatic downloadable drivers is to provide drivers that are NOT in 
>>>the kernel. If the user connects a new device and the system doesn't find a 
>>>driver for it it would request one from some server. This server could either 
>>>provide a driver itself, based the device's vendor and model id, or forward 
>>>the system to the vendor's server which would then try to find one for the 
>>>model.
>>>
>>But how do you resolve all of the previous points I raised about
>>compilers,
>>
>Just yesterday I had to replace gcc 3.0.1 with old but working
>egcs-2.91.66. The C download was just 1.5 MBytes, and C++ was
>only 1.5 MBytes more - though you wouldn't need C++ for the kernel.
>
>This means that gcc is just another dependency that the script will
>resolve before even trying to fetch and compile the driver. The
>size of the required download is acceptable even for modem users,
>and that has to be done only once.
>
>Of course, distributions can make the gcc (suitable for kernel
>compilations) a required (base) package.
>
I agree that a C compiler is a nice thing to have, but no one must be 
forced to have it (say, end users). If it becomes a "base package", 
every computer must have it installed and then people will start using 
it when it's not necessary (huge mess). The best thing to do here is to 
install the right version of gcc on demand (just as we do for drivers).

>>.config files,
>>
>I think they are available in "kernel-devel" package or something
>like that. They easily can be shipped as part of the kernel RPM.
>The kernel-devel package is already available in RH, for example.
>
>Of course, the kernel itself can contain a copy of its own .config
>file and export it through /proc fs, along with all compiler options,
>that would be neat :-) and only requires an optional module.
>
Standardize _both_ as the "current kernel configuration" 
(/usr/src/linux/.config _and_ a proc fs entry) so that driver sources 
can freely use it. I'd go for the first one though, here's the rationale:
Since we are going to allow third-party kernel drivers to be compiled 
for a kernel, we may as well grant them the right to do everything that 
a "normally" (in-kernel) distributed driver can do. A driver may depend 
on other drivers to compile/work and they will need to be downloaded, 
configured and installed as well. A driver must be allowed to set (its 
own, of course) entries in the kernel configuration file , modifying it 
for other third-party drivers to read. Thus it's better to use 
/usr/src/linux/.config .

>>in kernel apis,
>>
>That's not something new. APIs change all the time, everywhere, not
>only in OSes. Programmers just need to use #if ... #endif and maybe
>the driver should be marked as "suitable" or "tested" on some range
>of kernel releases.
>
Couldn't the kernel's configuration file be useful in solving this 
problem? API authors could use it to describe the changes they made in 
the API.

>>and security with regards to "automatic downloads"?
>>
>Linus already signs his kernel releases. There is no reason why
>driver maintainers can't join the web of trust. Then GPG will
>cheerfully accept signatures of partially trusted signers. It is
>very easy to do, publishers only need to get their public key
>signed by many other publishers, and the threshold for the sufficient
>trust can be hardcoded to be fairly high.
>
I wish I could find something wrong with that... Then I would have 
something to say here. :-)

>/***/
>
>It is clear to me that Linux needs a mechanism for fetching and
>installing 3rd party drivers. The proposed technology seems to
>be quite simple, and requires very little development because
>all components are already here. I think, a single script can
>definitely accomplish searching, downloading, compiling (kbuild?)
>and installing the driver. This project is all about organizing
>software and people to do it right.
>
Yep. Our slogan should be: "Do it when you have to!". :-)

This script can very well be a hotplug agent supporting four ACTIONs. I 
posted a message about this a few days ago, but since I didn't put any 
Subject line no one must have read it. Here it is again:

/* Quote myself: */

Well, a "configure" script would be good to use for source code. It
would return success/failure depending on whether the local
architecture/configuration is known to work for compiling the driver. If
it's not, then manual compilation and installation is required. Drivers
should be tested on as many kernel versions as possible and the
"configure" script should do any necessary tweakings to make the driver
compile on the particular system (as GNU apps do).

By "format" I didn't refer only to a file format that drivers are
packaged in but also to certain semantics -- like "how is this package
compiled and installed?". So, a driver that comes in source code in a
.tar.gz  and has a "configure" script that must be run before make is in
different "format" than a .tar.gz that doesn't need "configure".

Also, installation of a driver should have its mirror (uninstallation)
as well. Of course, it's not very practical to install/uninstall a
driver every time it's devices are plugged/unplugged, so it should be
left to the user to uninstall the driver when necessary.

pcimodules should do the high-level management for downloading and
installing a driver. It contacts the server to find out where the driver
is, downloads the "resource" and then passes control to a specific
"resource agent" which can install/uninstall this kind of resource.
Resource agents are placed in /etc/hotplug/driver and they accept:
ACTION={install,uninstall}
RESOURCE=/filename/of/resource.extension
TYPE={tarball,tarball_gnu,...}
That also suggests adding a new "driver.agent" to be called by
/sbin/hotplug .

If no one is willing to work on this, I'll try to set up some
client-side scripts on my localhost.





_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Thursday 18 October 2001 01:32, you wrote:
> But how do you resolve all of the previous points I raised about
> compilers, .config files, in kernel apis, and security with regards to
> "automatic downloads"?

- compilers

By requiring the user to have a certain gcc (make, ld..) version installed, 
possibly called kgcc or installed independently of the regular compiler. Its 
path must be be specified in a configuration file.

- .config files

The source, or at least the headers, and the .config files of the installed 
kernel must be available. The location could be specified in the 
configuration file as well. The only real problem are .config files that lack 
options that are needed by a driver (in other words the installed kernel does 
not have all features needed by the driver).

- kernel apis

Like with today's patches, even though it would be nice if source 
compatibility in the stable kernel series could be guaranteed.

- security

The security is not much different than with Debian's apt, Redhat Network or 
Suse's YOU. Either you get the driver from a trusted source, or the user has 
to be asked before the system downloads from a new/untrusted source.  To 
allow a download from new/untrusted sources you need to know the root 
password, of course (or maybe be in a special group)

bye...

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Keith Owens]

On Wed, 17 Oct 2001 17:07:32 -0700, 
Dmitri <dmitri@users.sourceforge.net> wrote:
>Of course, distributions can make the gcc (suitable for kernel
>compilations) a required (base) package.

That will be a requirement for any users who want to compile their own
or add modules onto existing kernels.

>> .config files,
>
>I think they are available in "kernel-devel" package or something
>like that. They easily can be shipped as part of the kernel RPM.
>The kernel-devel package is already available in RH, for example.

kbuild 2.5 introduces a standard method of installing .config and
System.map at the same time that the kernel and modules are installed.
The files do not have to be installed, but if they are, the default
location is /lib/modules/$(KERNELRELEASE)/{.config,System.map}.  Can
you tell that I have got fed up of distributions putting .config and
System.map in strange places?

>Of course, the kernel itself can contain a copy of its own .config
>file and export it through /proc fs, along with all compiler options,
>that would be neat :-) and only requires an optional module.

Proposed many times, refused by Linus each time.  It is kernel bloat
for a user space problem that can be solved in user space.  See kbuild 2.5.

>Linus already signs his kernel releases.

No he does not.  The signatures on files obtained from ftp.kernel.org
and mirrors are automatically generated by a script on kernel.org as
files are uploaded.  The signature is by ftpadmin, not the person who
put the file there.  It says nothing about who uploaded the file, it
only proves that the file came from kernel.org and has not been
tampered with since upload.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Dmitri]

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

Quoting Keith Owens <kaos@ocs.com.au>:

> >Linus already signs his kernel releases.
> 
> No he does not.  The signatures on files obtained from ftp.kernel.org
> and mirrors are automatically generated by a script on kernel.org as
> files are uploaded.  The signature is by ftpadmin, not the person who
> put the file there.  It says nothing about who uploaded the file, it
> only proves that the file came from kernel.org and has not been
> tampered with since upload.

The idea is the same: the owner of the secret key signs the package.
That owner can be a human or a machine. You have to trust the owner.
Of course, automated signing is less secure and I don't think it
can be used in this project.

In fact, what the ftp server can do is to verify the signature of the
package after the developer uploads it. Only properly signed files,
from known developers, are permitted to stay on the distribution
site. Just another little bit of security...

Dmitri

-- 
panic("esp: Aiee penguin on the SCSI-bus.");
  (from linux/drivers/scsi/esp.c)

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Automatic download and installation of drivers.

[Greg KH]

On Thu, Oct 18, 2001 at 01:19:47AM +0100, Stamatis Mitrofanis wrote:
> I agree that a C compiler is a nice thing to have, but no one must be 
> forced to have it (say, end users). If it becomes a "base package", 
> every computer must have it installed and then people will start using 
> it when it's not necessary (huge mess). The best thing to do here is to 
> install the right version of gcc on demand (just as we do for drivers).

And what version of gcc would you install?  Different versions for
different vendor kernels and even releases  :)

> Standardize _both_ as the "current kernel configuration" 
> (/usr/src/linux/.config _and_ a proc fs entry) so that driver sources 
> can freely use it. I'd go for the first one though, here's the rationale:
> Since we are going to allow third-party kernel drivers to be compiled 
> for a kernel, we may as well grant them the right to do everything that 
> a "normally" (in-kernel) distributed driver can do. A driver may depend 
> on other drivers to compile/work and they will need to be downloaded, 
> configured and installed as well. A driver must be allowed to set (its 
> own, of course) entries in the kernel configuration file , modifying it 
> for other third-party drivers to read. Thus it's better to use 
> /usr/src/linux/.config .

The /proc .config argument has been answered by Linus.  So you can't
guarantee that it will be available unless you apply a third party patch
to your kernel.

But I don't keep my kernels in /usr/src/linux, and neither do most
people.  And vendor kernels include a number of different .config files,
all of them placed in a different file within the .src.rpm.  Work with
the LSB if you want to push for this kind of standardization.

> 
> >>in kernel apis,
> >>
> >That's not something new. APIs change all the time, everywhere, not
> >only in OSes. Programmers just need to use #if ... #endif and maybe
> >the driver should be marked as "suitable" or "tested" on some range
> >of kernel releases.
> >
> Couldn't the kernel's configuration file be useful in solving this 
> problem? API authors could use it to describe the changes they made in 
> the API.

No, the .config file is used to describe the options that the user wants
built for the kernel.  It is not a language that can express API
changes.

How would you describe the fact that between kernels x and y, the struct
usb_serial_device changed the field "lock" from being a "struct
spinlock_t" to being a "struct semaphore"?  Changes that that _have_ to
have code changes to the driver made.  That's the main problem that you
are going to run into here.

> This script can very well be a hotplug agent supporting four ACTIONs. I 
> posted a message about this a few days ago, but since I didn't put any 
> Subject line no one must have read it. Here it is again:

Remember /sbin/hotplug runs as root.  You do _not_ want to be doing any
of these actions as root :)

Good luck,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Monday 29 October 2001 22:53, you wrote:
> But I don't keep my kernels in /usr/src/linux, and neither do most
> people.  And vendor kernels include a number of different .config files,
> all of them placed in a different file within the .src.rpm.  Work with
> the LSB if you want to push for this kind of standardization.

You don't have to use /usr/src/linux, the system would need its own kernel 
tree at different location (installed by the distribution or copied by a make 
target similar to modules_install when you compile the kernel yourself). You 
shouldn't even access it yourself.

For each kernel release there must be a gcc version defined, and the 
compilation system must have this gcc available. This could mean that you 
have 10 gccs (if you have 10 different kernels installed). These gcc's should 
not be in the usual PATH directories (/usr/bin..) but in a private directory.


> How would you describe the fact that between kernels x and y, the struct
> usb_serial_device changed the field "lock" from being a "struct
> spinlock_t" to being a "struct semaphore"?  Changes that that _have_ to
> have code changes to the driver made.  That's the main problem that you
> are going to run into here.

IMHO it is neccessary that stable kernels keep source compatibility.


> Remember /sbin/hotplug runs as root.  You do _not_ want to be doing any
> of these actions as root :)

But it shouldn't be a problem to su to a special user for compiling.

bye...

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Tue, Oct 30, 2001 at 09:26:48AM +0100, Tim Jansen wrote:
> 
> For each kernel release there must be a gcc version defined, and the 
> compilation system must have this gcc available. This could mean that you 
> have 10 gccs (if you have 10 different kernels installed). These gcc's should 
> not be in the usual PATH directories (/usr/bin..) but in a private directory.

Where do you define this version of gcc?  Many versions of gcc can
successfully compile a kernel.

> IMHO it is neccessary that stable kernels keep source compatibility.

That is not way Linux kernel development is done, sorry.

thanks,

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Tim Jansen]

On Tuesday 30 October 2001 18:21, Greg KH wrote:
> Where do you define this version of gcc?  
> Many versions of gcc can successfully compile a kernel.

The easiest way would be to include a file in the kernel that contains the 
version number of one compiler that is able to compile the kernel. This 
compiler is then used to compile the kernel.


> > IMHO it is neccessary that stable kernels keep source compatibility.
> That is not way Linux kernel development is done, sorry.

If the driver does not compile and the package containing the driver states 
that the driver has not been tested with the kernel version it can just abort 
and tell the user that the driver has not been ported the newer version yet. 
This is not worse than the current situation for both user and developer.

I wonder for how long it is possible to keep all existing drivers in the 
kernel source though. 

bye...

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Dmitri]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

Quoting Tim Jansen <tim@tjansen.de>:

> On Tuesday 30 October 2001 18:21, Greg KH wrote:
> > Where do you define this version of gcc?  
> > Many versions of gcc can successfully compile a kernel.
> 
> The easiest way would be to include a file in the kernel that contains the 
> version number of one compiler that is able to compile the kernel. This 
> compiler is then used to compile the kernel.

This will also make the system compiler-independent. You may have
the most modern gcc 3.0.1 for your own compilations, but the kernel
components will be built with tried, tested and true gcc 2.96.x
(or whatever).

Dmitri

-- 
Linux: Where Don't We Want To Go Today?
  (Submitted by Pancrazio De Mauro, paraphrasing some well-known sales talk.)

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Automatic download and installation of drivers.

[Keith Owens]

On Mon, 29 Oct 2001 13:53:40 -0800, 
Greg KH <greg@kroah.com> wrote:
>On Thu, Oct 18, 2001 at 01:19:47AM +0100, Stamatis Mitrofanis wrote:
>> I agree that a C compiler is a nice thing to have, but no one must be 
>> forced to have it (say, end users). If it becomes a "base package", 
>> every computer must have it installed and then people will start using 
>> it when it's not necessary (huge mess). The best thing to do here is to 
>> install the right version of gcc on demand (just as we do for drivers).
>
>And what version of gcc would you install?  Different versions for
>different vendor kernels and even releases  :)

I have decided to add CONFIG_KBUILD_GCC_VERSION to kbuild 2.5, it
records the values of gcc -b and -V used to build the kernel.  The
string is automatically generated and added to the .config, _every_
source file will implicitly depend on that config variable so changing
the compiler will force a complete recompile.

Building the kernel and modules from a mixture of compilers is not a
supported operation.

This is almost true in kbuild 2.5 already.  If you compile with 'gcc'
then recompile with 'kgcc' or switch from 'gcc' to 'gcc -V 3.0.1',
everything gets rebuilt because the compiler name and flags are part of
the saved command, any change in a command forces a recompile of the
affected objects.  Adding the CONFIG_ variable will document which
compiler was used and catches the pathological case where 'gcc' is used
but it points to different compilers on different days.

Version information for other build utilities will be added as required.

>But I don't keep my kernels in /usr/src/linux, and neither do most
>people.  And vendor kernels include a number of different .config files,
>all of them placed in a different file within the .src.rpm.  Work with
>the LSB if you want to push for this kind of standardization.

Or the kbuild maintainer ;).  LSB does not document down to this level
of detail so I am standardizing where .config and other files go in
2.5.  /lib/modules/`uname -r`/{.config,System.map,trees}.  kbuild 2.5
is not restricted to a single source tree so the single build symlink
is useless in 2.5, instead the 'trees' file defines the source tree(s)
and the object tree.


_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Automatic download and installation of drivers.

[Greg KH]

On Wed, Oct 31, 2001 at 11:36:05PM +1100, Keith Owens wrote:
> 
> Or the kbuild maintainer ;).  LSB does not document down to this level
> of detail so I am standardizing where .config and other files go in
> 2.5.  /lib/modules/`uname -r`/{.config,System.map,trees}.  kbuild 2.5
> is not restricted to a single source tree so the single build symlink
> is useless in 2.5, instead the 'trees' file defines the source tree(s)
> and the object tree.

Very nice.  I am sure lots of people will appreciate this and the gcc
config item.  Thanks for adding them.

greg k-h

_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel