PCI/USB Vendor and model from database

PCI/USB Vendor and model from database

[José Félix Ontañón]

Hi, everybody!

Maybe this is kinda silly question but, I wonder why isn't
"ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
imported by default using the pretty /lib/udev/usb-db and
/lib/udev/pci-db commands for every founded pci/usb device?

I mean, by adding this simple rules:

SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"

I think this properties are very useful for apps that discover
hardware through udev, so they will not have to reinvent the wheel by
finding vendor/product names by themselves. Don't you think?

Cheers!

--
http://fontanon.org

Re: PCI/USB Vendor and model from database

[Greg KH]

On Tue, Dec 28, 2010 at 02:14:38PM +0100, José Félix Ontañón wrote:
> Hi, everybody!
> 
> Maybe this is kinda silly question but, I wonder why isn't
> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
> imported by default using the pretty /lib/udev/usb-db and
> /lib/udev/pci-db commands for every founded pci/usb device?
> 
> I mean, by adding this simple rules:
> 
> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
> 
> I think this properties are very useful for apps that discover
> hardware through udev, so they will not have to reinvent the wheel by
> finding vendor/product names by themselves. Don't you think?

That would slow down every device being found, for no real user at this
time, right?

How hard is it to use libpci for the userspace program to find this
information, if it really needs it?  That way, you also don't have to
have the database at early boot time, which keeps things simpler.

thanks,

greg k-h

Re: PCI/USB Vendor and model from database

[Lennart Poettering]

On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:

> Hi, everybody!
> 
> Maybe this is kinda silly question but, I wonder why isn't
> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
> imported by default using the pretty /lib/udev/usb-db and
> /lib/udev/pci-db commands for every founded pci/usb device?
> 
> I mean, by adding this simple rules:
> 
> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
> 
> I think this properties are very useful for apps that discover
> hardware through udev, so they will not have to reinvent the wheel by
> finding vendor/product names by themselves. Don't you think?

The database lookup is a linear search. As long as we invoke it only for
a small subset of devices that doesn't really matter much. But if we
start to look it up for every device we should probably spend the time
to improve the db lookup first.

It's simply a question of efficiency.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.

Re: PCI/USB Vendor and model from database

[Scott James Remnant]

2010/12/28 Lennart Poettering <lennart@poettering.net>:
> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
>
>> Hi, everybody!
>>
>> Maybe this is kinda silly question but, I wonder why isn't
>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
>> imported by default using the pretty /lib/udev/usb-db and
>> /lib/udev/pci-db commands for every founded pci/usb device?
>>
>> I mean, by adding this simple rules:
>>
>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
>>
>> I think this properties are very useful for apps that discover
>> hardware through udev, so they will not have to reinvent the wheel by
>> finding vendor/product names by themselves. Don't you think?
>
> The database lookup is a linear search. As long as we invoke it only for
> a small subset of devices that doesn't really matter much. But if we
> start to look it up for every device we should probably spend the time
> to improve the db lookup first.
>
> It's simply a question of efficiency.
>
One thing that might be interesting is if we could do the lookup at
the point of query, then store the answer back in the db for others to
use later.

That way we're efficient if nobody cares, and lose that efficiency
when they do (but only for the devices they query), and subsequently
efficient because the answer is cached.

This veers a bit dangerously towards having apps that link with
libudev process rules though?

Scott

Re: PCI/USB Vendor and model from database

[José Félix Ontañón]

El día 29 de diciembre de 2010 01:18, Scott James Remnant
<scott@netsplit.com> escribió:
> 2010/12/28 Lennart Poettering <lennart@poettering.net>:
>> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
>>
>>> Hi, everybody!
>>>
>>> Maybe this is kinda silly question but, I wonder why isn't
>>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
>>> imported by default using the pretty /lib/udev/usb-db and
>>> /lib/udev/pci-db commands for every founded pci/usb device?
>>>
>>> I mean, by adding this simple rules:
>>>
>>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
>>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
>>>
>>> I think this properties are very useful for apps that discover
>>> hardware through udev, so they will not have to reinvent the wheel by
>>> finding vendor/product names by themselves. Don't you think?
>>
>> The database lookup is a linear search. As long as we invoke it only for
>> a small subset of devices that doesn't really matter much. But if we
>> start to look it up for every device we should probably spend the time
>> to improve the db lookup first.
>>
>> It's simply a question of efficiency.
>>
> One thing that might be interesting is if we could do the lookup at
> the point of query, then store the answer back in the db for others to
> use later.
>
> That way we're efficient if nobody cares, and lose that efficiency
> when they do (but only for the devices they query), and subsequently
> efficient because the answer is cached.
>
> This veers a bit dangerously towards having apps that link with
> libudev process rules though?
>
> Scott
>

First thing coming to my mind: there's not enough pci/usb devices
present on the system for increasing the computational cost
significantly. Querying the vendor/model names will be performed only
one time, at udev running, and then only one time on "add" action for
this subsystems. Am i wrong?

Anyway, taking care about the efficiency, the Scott solutions sounds
good to me but implies development on udev directly. I'm writing some
apps using udev so, in the meantime, will be really nice if some more
questions could be answered:

Is it acceptable to add the mentioned udev rule for a system
application using udev? I mean, i.e. udisk add some rules for
importing some props. And for a desktop app? Is it acceptable too?

I think it's quite comfortable, for me as desktop app coder, to
delegate on udev the vendor/model name querying, more if we think udev
provides the cool pci-db and usb-db commands.

Cheers!

-- 
http://fontanon.org

Re: PCI/USB Vendor and model from database

[Greg KH]

On Wed, Dec 29, 2010 at 05:08:56PM +0100, José Félix Ontañón wrote:
> El día 29 de diciembre de 2010 01:18, Scott James Remnant
> <scott@netsplit.com> escribió:
> > 2010/12/28 Lennart Poettering <lennart@poettering.net>:
> >> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
> >>
> >>> Hi, everybody!
> >>>
> >>> Maybe this is kinda silly question but, I wonder why isn't
> >>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
> >>> imported by default using the pretty /lib/udev/usb-db and
> >>> /lib/udev/pci-db commands for every founded pci/usb device?
> >>>
> >>> I mean, by adding this simple rules:
> >>>
> >>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
> >>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
> >>>
> >>> I think this properties are very useful for apps that discover
> >>> hardware through udev, so they will not have to reinvent the wheel by
> >>> finding vendor/product names by themselves. Don't you think?
> >>
> >> The database lookup is a linear search. As long as we invoke it only for
> >> a small subset of devices that doesn't really matter much. But if we
> >> start to look it up for every device we should probably spend the time
> >> to improve the db lookup first.
> >>
> >> It's simply a question of efficiency.
> >>
> > One thing that might be interesting is if we could do the lookup at
> > the point of query, then store the answer back in the db for others to
> > use later.
> >
> > That way we're efficient if nobody cares, and lose that efficiency
> > when they do (but only for the devices they query), and subsequently
> > efficient because the answer is cached.
> >
> > This veers a bit dangerously towards having apps that link with
> > libudev process rules though?
> >
> > Scott
> >
> 
> First thing coming to my mind: there's not enough pci/usb devices
> present on the system for increasing the computational cost
> significantly.

How can you say that?  I see systems with thousands of PCI devices on it
as being very common in some areas.

> I think it's quite comfortable, for me as desktop app coder, to
> delegate on udev the vendor/model name querying, more if we think udev
> provides the cool pci-db and usb-db commands.

Again, what's wrong with using libpci directly for this?  That's what it
is there for.

thanks,

greg k-h

Re: PCI/USB Vendor and model from database

[Kay Sievers]

2010/12/29 José Félix Ontañón <felixonta@gmail.com>:
> El día 29 de diciembre de 2010 01:18, Scott James Remnant
> <scott@netsplit.com> escribió:
>> 2010/12/28 Lennart Poettering <lennart@poettering.net>:
>>> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
>>>>
>>>> Maybe this is kinda silly question but, I wonder why isn't
>>>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
>>>> imported by default using the pretty /lib/udev/usb-db and
>>>> /lib/udev/pci-db commands for every founded pci/usb device?
>>>>
>>>> I mean, by adding this simple rules:
>>>>
>>>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
>>>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
>>>>
>>>> I think this properties are very useful for apps that discover
>>>> hardware through udev, so they will not have to reinvent the wheel by
>>>> finding vendor/product names by themselves. Don't you think?
>>>
>>> The database lookup is a linear search. As long as we invoke it only for
>>> a small subset of devices that doesn't really matter much. But if we
>>> start to look it up for every device we should probably spend the time
>>> to improve the db lookup first.
>>>
>>> It's simply a question of efficiency.
>>>
>> One thing that might be interesting is if we could do the lookup at
>> the point of query, then store the answer back in the db for others to
>> use later.
>>
>> That way we're efficient if nobody cares, and lose that efficiency
>> when they do (but only for the devices they query), and subsequently
>> efficient because the answer is cached.
>>
>> This veers a bit dangerously towards having apps that link with
>> libudev process rules though?

> First thing coming to my mind: there's not enough pci/usb devices
> present on the system for increasing the computational cost
> significantly. Querying the vendor/model names will be performed only
> one time, at udev running, and then only one time on "add" action for
> this subsystems. Am i wrong?

As Greg says, it really can be a lot of devices, and the linear search
in a huge text file isn't exactly what we want to do. We might need a
real indexed database. Ideally one that does not only contain human
readable identifiers but also whatever meta-data which is useful to
users.

> Anyway, taking care about the efficiency, the Scott solutions sounds
> good to me but implies development on udev directly. I'm writing some
> apps using udev so, in the meantime, will be really nice if some more
> questions could be answered:

It's difficult. We run in untrusted user-context, and can not update
the udev database. It would be a major change in the architecture,
which I'm not sure we should get into that. For many of the devices
on-demand will not be sufficient, as we need the symlinks, tags and
other things.

> Is it acceptable to add the mentioned udev rule for a system
> application using udev? I mean, i.e. udisk add some rules for
> importing some props. And for a desktop app? Is it acceptable too?

Sure, you can do that. Just check that it isn't already run before your rules.

> I think it's quite comfortable, for me as desktop app coder, to
> delegate on udev the vendor/model name querying, more if we think udev
> provides the cool pci-db and usb-db commands.

As Greg already pointed out, you should probably look up the stuff you
need, yourself in the database files.

Udev must not become the new HAL. We need to be very very careful here. :)

Kay

Re: PCI/USB Vendor and model from database

[José Félix Ontañón]

El día 29 de diciembre de 2010 17:16, Greg KH <greg@kroah.com> escribió:
> On Wed, Dec 29, 2010 at 05:08:56PM +0100, José Félix Ontañón wrote:
>> El día 29 de diciembre de 2010 01:18, Scott James Remnant
>> <scott@netsplit.com> escribió:
>> > 2010/12/28 Lennart Poettering <lennart@poettering.net>:
>> >> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
>> >>
>> >>> Hi, everybody!
>> >>>
>> >>> Maybe this is kinda silly question but, I wonder why isn't
>> >>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
>> >>> imported by default using the pretty /lib/udev/usb-db and
>> >>> /lib/udev/pci-db commands for every founded pci/usb device?
>> >>>
>> >>> I mean, by adding this simple rules:
>> >>>
>> >>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
>> >>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
>> >>>
>> >>> I think this properties are very useful for apps that discover
>> >>> hardware through udev, so they will not have to reinvent the wheel by
>> >>> finding vendor/product names by themselves. Don't you think?
>> >>
>> >> The database lookup is a linear search. As long as we invoke it only for
>> >> a small subset of devices that doesn't really matter much. But if we
>> >> start to look it up for every device we should probably spend the time
>> >> to improve the db lookup first.
>> >>
>> >> It's simply a question of efficiency.
>> >>
>> > One thing that might be interesting is if we could do the lookup at
>> > the point of query, then store the answer back in the db for others to
>> > use later.
>> >
>> > That way we're efficient if nobody cares, and lose that efficiency
>> > when they do (but only for the devices they query), and subsequently
>> > efficient because the answer is cached.
>> >
>> > This veers a bit dangerously towards having apps that link with
>> > libudev process rules though?
>> >
>> > Scott
>> >
>>
>> First thing coming to my mind: there's not enough pci/usb devices
>> present on the system for increasing the computational cost
>> significantly.
>
> How can you say that?  I see systems with thousands of PCI devices on it
> as being very common in some areas.

You're right, I was to fast thinking about personal_computers-like
only. That's because I said it maybe was a silly question, sorry.

>> I think it's quite comfortable, for me as desktop app coder, to
>> delegate on udev the vendor/model name querying, more if we think udev
>> provides the cool pci-db and usb-db commands.
>
> Again, what's wrong with using libpci directly for this?  That's what it
> is there for.

Greg, i'm coding python, do you know about a python bindings for
libpci? It would be really nice if you could tell me because I didn't
found.

Thanks for your answers.

> thanks,
>
> greg k-h
>



-- 
http://fontanon.org

Re: PCI/USB Vendor and model from database

[José Félix Ontañón]

El día 29 de diciembre de 2010 17:34, Kay Sievers
<kay.sievers@vrfy.org> escribió:
> 2010/12/29 José Félix Ontañón <felixonta@gmail.com>:
>> El día 29 de diciembre de 2010 01:18, Scott James Remnant
>> <scott@netsplit.com> escribió:
>>> 2010/12/28 Lennart Poettering <lennart@poettering.net>:
>>>> On Tue, 28.12.10 14:14, José Félix Ontañón (felixonta@gmail.com) wrote:
>>>>>
>>>>> Maybe this is kinda silly question but, I wonder why isn't
>>>>> "ID_VENDOR_FROM_DATABASE / ID_MODEL_FROM_DATABASE"-like properties
>>>>> imported by default using the pretty /lib/udev/usb-db and
>>>>> /lib/udev/pci-db commands for every founded pci/usb device?
>>>>>
>>>>> I mean, by adding this simple rules:
>>>>>
>>>>> SUBSYSTEMS="usb", IMPORT{program}="usb-db %p"
>>>>> SUBSYSTEMS="pci", IMPORT{program}="pci-db %p"
>>>>>
>>>>> I think this properties are very useful for apps that discover
>>>>> hardware through udev, so they will not have to reinvent the wheel by
>>>>> finding vendor/product names by themselves. Don't you think?
>>>>
>>>> The database lookup is a linear search. As long as we invoke it only for
>>>> a small subset of devices that doesn't really matter much. But if we
>>>> start to look it up for every device we should probably spend the time
>>>> to improve the db lookup first.
>>>>
>>>> It's simply a question of efficiency.
>>>>
>>> One thing that might be interesting is if we could do the lookup at
>>> the point of query, then store the answer back in the db for others to
>>> use later.
>>>
>>> That way we're efficient if nobody cares, and lose that efficiency
>>> when they do (but only for the devices they query), and subsequently
>>> efficient because the answer is cached.
>>>
>>> This veers a bit dangerously towards having apps that link with
>>> libudev process rules though?
>
>> First thing coming to my mind: there's not enough pci/usb devices
>> present on the system for increasing the computational cost
>> significantly. Querying the vendor/model names will be performed only
>> one time, at udev running, and then only one time on "add" action for
>> this subsystems. Am i wrong?
>
> As Greg says, it really can be a lot of devices, and the linear search
> in a huge text file isn't exactly what we want to do. We might need a
> real indexed database. Ideally one that does not only contain human
> readable identifiers but also whatever meta-data which is useful to
> users.
>
>> Anyway, taking care about the efficiency, the Scott solutions sounds
>> good to me but implies development on udev directly. I'm writing some
>> apps using udev so, in the meantime, will be really nice if some more
>> questions could be answered:
>
> It's difficult. We run in untrusted user-context, and can not update
> the udev database. It would be a major change in the architecture,
> which I'm not sure we should get into that. For many of the devices
> on-demand will not be sufficient, as we need the symlinks, tags and
> other things.

Ok, I see the problem, Kay. Lots of design for a system that pretends
to be small, isn't?

>> Is it acceptable to add the mentioned udev rule for a system
>> application using udev? I mean, i.e. udisk add some rules for
>> importing some props. And for a desktop app? Is it acceptable too?
>
> Sure, you can do that. Just check that it isn't already run before your rules.

I hope you don't mind if I tell the whole history now:

I'm working on a gnome-device-manager like app[1] (hal/dbus based). By
now it just pretends to be an user-friendly version of Lennart
Poettering udev-browse (browsing /sys as a treeview). I've realized
linux users tends to use lshw/lspci/lsusb for checking the
vendor_id/product_id and vendor/model names, so i though it could be
useful if a gnome-device-manager like app tell them.

After some of your and Greg's comments I think there's no doubt adding
the udev rules for importing the PCI/USB Vendor and model from
database it's not a good idea.

So, talking about efficiency, is it better to use "libwhatever" for
querying for the vendor/model names everytime the user launchs the
app?

[1] http://gitorious.org/udev-discover

>> I think it's quite comfortable, for me as desktop app coder, to
>> delegate on udev the vendor/model name querying, more if we think udev
>> provides the cool pci-db and usb-db commands.
>
> As Greg already pointed out, you should probably look up the stuff you
> need, yourself in the database files.

Well, and that points to my prev question. Is it acceptable to look up
the database files everytime the user runs the app? Not my intention
to appear lazy but I see myself doing nothing about it or coding a
vendor/model names cache or something alike, and i wonder if there's
no other way ...

> Udev must not become the new HAL. We need to be very very careful here. :)

Absolutely. I'm concerned about the problem.

> Kay

Thanks for your gentle answers Kay.

-- 
http://fontanon.org