WMI and Kernel:User interface

WMI and Kernel:User interface

[Darren Hart]

Linus and Greg,

We are in the process of redesigning the Windows Management Instrumentation
(WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
while allowing the kernel to filter requests that conflict with its own usage.
We'd like your take on how this approach relates to our commitment to not break
userspace.

For this discussion, we are specifically referring to ACPI PNP0C14 WMI
devices, consisting of a GUID and a set of methods and events, as well as a
precompiled intermediate description of the methods and arguments (MOF). Exposed
to userspace, these methods provide for BIOS interaction and are used for system
management as well as LEDs, hot keys, radio switches, etc. There is vendor
interest in achieving feature parity with Windows by exposing WMI methods to
userspace for system management.

While it appears WMI intended to be accessed from userspace, we have
made use of it in the kernel to support various laptop features by connecting
the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
challenge is continuing to use WMI for these platform features, while allowing
userspace to use it for system management tasks. Unfortunately, the WMI methods
are not guaranteed to be split up along granular functional lines, and we will
certainly face situations where the same GUID::METHOD_ID will be needed for a
kernel feature (say LED support) as well as a system management task.

To address this, I have proposed [3] that exporting WMI be opt-in, only done at
the request of and in collaboration with a vendor, with the kernel platform
driver given the opportunity to filter requests. This filtering would need to be
at the method and argument inspection level, such as checking for specific bits
in the input buffer, and rejecting the request if they conflict with an in
kernel usage (that's worst case, in some cases just GUID or method ID could be
sufficient).

Because the kernel and the platform drivers are under continual development, and
new systems appear regularly, we will encounter necessary changes to the
platform driver WMI request filters. These changes could be considered a change
to the kernel provided WMI interface to userspace. For example, we could
regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
later deny the call when we determine it interferes with kernel usage.

In your view, is it acceptable to provide a chardev interface, for example,
exposing WMI methods to userspace, with the understanding that the kernel may
choose to filter certain requests which conflict with its own use? And that this
filtering may change as new features are added to the platform drivers?

1. https://msdn.microsoft.com/en-us/library/aa384642(v=vs.85).aspx
2. https://lwn.net/Articles/391230/
3. https://lists.gt.net/linux/kernel/2671309/?page=1;

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> Linus and Greg,
> 
> We are in the process of redesigning the Windows Management Instrumentation
> (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
> Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
> while allowing the kernel to filter requests that conflict with its own usage.
> We'd like your take on how this approach relates to our commitment to not break
> userspace.
> 
> For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> devices, consisting of a GUID and a set of methods and events, as well as a
> precompiled intermediate description of the methods and arguments (MOF). Exposed
> to userspace, these methods provide for BIOS interaction and are used for system
> management as well as LEDs, hot keys, radio switches, etc. There is vendor
> interest in achieving feature parity with Windows by exposing WMI methods to
> userspace for system management.
> 
> While it appears WMI intended to be accessed from userspace, we have
> made use of it in the kernel to support various laptop features by connecting
> the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> challenge is continuing to use WMI for these platform features, while allowing
> userspace to use it for system management tasks. Unfortunately, the WMI methods
> are not guaranteed to be split up along granular functional lines, and we will
> certainly face situations where the same GUID::METHOD_ID will be needed for a
> kernel feature (say LED support) as well as a system management task.
> 
> To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> the request of and in collaboration with a vendor, with the kernel platform
> driver given the opportunity to filter requests. This filtering would need to be
> at the method and argument inspection level, such as checking for specific bits
> in the input buffer, and rejecting the request if they conflict with an in
> kernel usage (that's worst case, in some cases just GUID or method ID could be
> sufficient).
> 
> Because the kernel and the platform drivers are under continual development, and
> new systems appear regularly, we will encounter necessary changes to the
> platform driver WMI request filters. These changes could be considered a change
> to the kernel provided WMI interface to userspace. For example, we could
> regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> later deny the call when we determine it interferes with kernel usage.
> 
> In your view, is it acceptable to provide a chardev interface, for example,
> exposing WMI methods to userspace, with the understanding that the kernel may
> choose to filter certain requests which conflict with its own use? And that this
> filtering may change as new features are added to the platform drivers?

So, for example, if a new driver for a "brightness key" were added to
the kernel, all of a sudden the "raw" access to the wmi data through the
chardev would filtered away by the kernel and not seen by userspace?

Why would you want to do that?  What's wrong with providing "raw" access
through a chardev, and the current in-kernel access as well at the same
time?

I don't really understand what would "break" over time here.

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Darren Hart]

On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > Linus and Greg,
> > 
> > We are in the process of redesigning the Windows Management Instrumentation
> > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
> > Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
> > while allowing the kernel to filter requests that conflict with its own usage.
> > We'd like your take on how this approach relates to our commitment to not break
> > userspace.
> > 
> > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > devices, consisting of a GUID and a set of methods and events, as well as a
> > precompiled intermediate description of the methods and arguments (MOF). Exposed
> > to userspace, these methods provide for BIOS interaction and are used for system
> > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > interest in achieving feature parity with Windows by exposing WMI methods to
> > userspace for system management.
> > 
> > While it appears WMI intended to be accessed from userspace, we have
> > made use of it in the kernel to support various laptop features by connecting
> > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > challenge is continuing to use WMI for these platform features, while allowing
> > userspace to use it for system management tasks. Unfortunately, the WMI methods
> > are not guaranteed to be split up along granular functional lines, and we will
> > certainly face situations where the same GUID::METHOD_ID will be needed for a
> > kernel feature (say LED support) as well as a system management task.
> > 
> > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > the request of and in collaboration with a vendor, with the kernel platform
> > driver given the opportunity to filter requests. This filtering would need to be
> > at the method and argument inspection level, such as checking for specific bits
> > in the input buffer, and rejecting the request if they conflict with an in
> > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > sufficient).
> > 
> > Because the kernel and the platform drivers are under continual development, and
> > new systems appear regularly, we will encounter necessary changes to the
> > platform driver WMI request filters. These changes could be considered a change
> > to the kernel provided WMI interface to userspace. For example, we could
> > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > later deny the call when we determine it interferes with kernel usage.
> > 
> > In your view, is it acceptable to provide a chardev interface, for example,
> > exposing WMI methods to userspace, with the understanding that the kernel may
> > choose to filter certain requests which conflict with its own use? And that this
> > filtering may change as new features are added to the platform drivers?
> 
> So, for example, if a new driver for a "brightness key" were added to
> the kernel, all of a sudden the "raw" access to the wmi data through the
> chardev would filtered away by the kernel and not seen by userspace?
> 

Pali provided some detail in the rather lengthy thread I linked to [3], but the
summary is that there is nothing to encourage vendors to separate out
functionality into separate method ids. Some do, the asus-wmi driver seems to
have a reasonable separation, others do a lot with a single method id.

In the scenario you mention, it could be that the brightness key shows up in a
new laptop. Because we've never seen it before, the kernel driver doesn't detect
it and send the input event. The clever user realizes they can write a userspace
program to deal with this [a] by writing a my-laptop-brightness-key-daemon which
also makes a WMI method call to affect the change in brightness in response to
the keypress.

Another user notices the dmesg is reporting "Unknown WMI Event" when they press
that key and send a patch to make the corresponding wmi call to affect the
brightness change (which depends on driver data to track the brightness value).
To avoid conflicting with userspace, they also blacklist the WMI_BRIGHTNESS_CMD
from being called by userspace.

Now the first user makes a call to set the brightness, and receives an error
code instead of a successful response. Their program no longer works - although
with a current kernel, their brightness key would.

I glossed over a few things there, but the description isn't too far off from a
plausible scenario.

> Why would you want to do that?  What's wrong with providing "raw" access
> through a chardev, and the current in-kernel access as well at the same
> time?
> 
> I don't really understand what would "break" over time here.

Like the scenario above, if the behavior has state, if userspace and the kernel
attempt to control it, neither will have an accurate state representation.

We could allow both and if things start not working, the user would have to
choose between the kernel driver and the userspace management application. This
was the scenario Pali was particularly keen to avoid.

That said, to date I haven't come across anything that states there can only be
one userspace application accessing WMI at any given time. It should be
straightforward to serialize WMI calls such that they cannot "cross the
streams".

> 
> thanks,
> 
> greg k-h
> 

a. This is perhaps a contrived scenario since hotkeys generate events, and as
far as I understand it, there is no interest in exporting events to userspace,
just the methods.

-- 
Darren Hart
VMware Open Source Technology Center

RE: WMI and Kernel:User interface

[Mario.Limonciello]

> -----Original Message-----
> From: Darren Hart [mailto:dvhart@infradead.org]
> Sent: Wednesday, May 10, 2017 1:12 AM
> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>; Limonciello, Mario
> <Mario_Limonciello@Dell.com>; Pali Rohár <pali.rohar@gmail.com>; Andy
> Shevchenko <andriy.shevchenko@linux.intel.com>; Rafael Wysocki
> <rjw@rjwysocki.net>; Andy Lutomirski <luto@amacapital.net>; LKML <linux-
> kernel@vger.kernel.org>; platform-driver-x86@vger.kernel.org
> Subject: Re: WMI and Kernel:User interface
> 
> On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > Linus and Greg,
> > >
> > > We are in the process of redesigning the Windows Management
> Instrumentation
> > > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-
> Based
> > > Enterprise Management (WBEM). We are looking to provide WMI access to
> userspace,
> > > while allowing the kernel to filter requests that conflict with its own usage.
> > > We'd like your take on how this approach relates to our commitment to not
> break
> > > userspace.
> > >
> > > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > > devices, consisting of a GUID and a set of methods and events, as well as a
> > > precompiled intermediate description of the methods and arguments (MOF).
> Exposed
> > > to userspace, these methods provide for BIOS interaction and are used for
> system
> > > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > > interest in achieving feature parity with Windows by exposing WMI methods to
> > > userspace for system management.
> > >
> > > While it appears WMI intended to be accessed from userspace, we have
> > > made use of it in the kernel to support various laptop features by connecting
> > > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > > challenge is continuing to use WMI for these platform features, while allowing
> > > userspace to use it for system management tasks. Unfortunately, the WMI
> methods
> > > are not guaranteed to be split up along granular functional lines, and we will
> > > certainly face situations where the same GUID::METHOD_ID will be needed for
> a
> > > kernel feature (say LED support) as well as a system management task.
> > >
> > > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > > the request of and in collaboration with a vendor, with the kernel platform
> > > driver given the opportunity to filter requests. This filtering would need to be
> > > at the method and argument inspection level, such as checking for specific bits
> > > in the input buffer, and rejecting the request if they conflict with an in
> > > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > > sufficient).
> > >
> > > Because the kernel and the platform drivers are under continual development,
> and
> > > new systems appear regularly, we will encounter necessary changes to the
> > > platform driver WMI request filters. These changes could be considered a
> change
> > > to the kernel provided WMI interface to userspace. For example, we could
> > > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > > later deny the call when we determine it interferes with kernel usage.
> > >
> > > In your view, is it acceptable to provide a chardev interface, for example,
> > > exposing WMI methods to userspace, with the understanding that the kernel
> may
> > > choose to filter certain requests which conflict with its own use? And that this
> > > filtering may change as new features are added to the platform drivers?
> >
> > So, for example, if a new driver for a "brightness key" were added to
> > the kernel, all of a sudden the "raw" access to the wmi data through the
> > chardev would filtered away by the kernel and not seen by userspace?
> >
> 
> Pali provided some detail in the rather lengthy thread I linked to [3], but the
> summary is that there is nothing to encourage vendors to separate out
> functionality into separate method ids. Some do, the asus-wmi driver seems to
> have a reasonable separation, others do a lot with a single method id.
> 
> In the scenario you mention, it could be that the brightness key shows up in a
> new laptop. Because we've never seen it before, the kernel driver doesn't detect
> it and send the input event. The clever user realizes they can write a userspace
> program to deal with this [a] by writing a my-laptop-brightness-key-daemon which
> also makes a WMI method call to affect the change in brightness in response to
> the keypress.
> 
> Another user notices the dmesg is reporting "Unknown WMI Event" when they
> press
> that key and send a patch to make the corresponding wmi call to affect the
> brightness change (which depends on driver data to track the brightness value).
> To avoid conflicting with userspace, they also blacklist the WMI_BRIGHTNESS_CMD
> from being called by userspace.
> 
> Now the first user makes a call to set the brightness, and receives an error
> code instead of a successful response. Their program no longer works - although
> with a current kernel, their brightness key would.
> 
> I glossed over a few things there, but the description isn't too far off from a
> plausible scenario.
> 
> > Why would you want to do that?  What's wrong with providing "raw" access
> > through a chardev, and the current in-kernel access as well at the same
> > time?
> >
> > I don't really understand what would "break" over time here.
> 
> Like the scenario above, if the behavior has state, if userspace and the kernel
> attempt to control it, neither will have an accurate state representation.
> 
> We could allow both and if things start not working, the user would have to
> choose between the kernel driver and the userspace management application. This
> was the scenario Pali was particularly keen to avoid.
> 
> That said, to date I haven't come across anything that states there can only be
> one userspace application accessing WMI at any given time. It should be
> straightforward to serialize WMI calls such that they cannot "cross the
> streams".
> 
> >
> > thanks,
> >
> > greg k-h
> >
> 
> a. This is perhaps a contrived scenario since hotkeys generate events, and as
> far as I understand it, there is no interest in exporting events to userspace,
> just the methods.
> 
> --
> Darren Hart
> VMware Open Source Technology Center

This above discussion is confusing because it's referring specifically to "WMI events".
related to brightness keypresses that don't make sense to go to userspace.

The more interesting (and potentially problematic) area is things that read and write
data using ASL methods.

So here's a "more" realistic scenario:

OEM has support through a WMI function to control keyboard backlight timeouts
and intensity.  That same WMI function also can support turning on/off an individual
USB port.  Backlight timeouts are done by setting the first argument to "1" and USB
port control is done by setting first argument to "2".

Some userspace app is developed that can control both of these functions through
the chardev.  Later an enterprising young kernel developer realizes that backlight 
control should be done through a platform driver instead.

They write a platform driver to do it, and add a filter to block "1" arguments from
userspace.  Now if the userspace app tries to call the chardev with the "1" argument
some error code is returned indicating this request is not supported.

The result is the userspace app broke, but it broke because the kernel is supporting
the method in a much smarter and more scalable way.

Re: WMI and Kernel:User interface

[Darren Hart]

On Wed, May 10, 2017 at 10:02:46PM +0000, Mario.Limonciello@dell.com wrote:
> > -----Original Message-----
> > From: Darren Hart [mailto:dvhart@infradead.org]
> > Sent: Wednesday, May 10, 2017 1:12 AM
> > To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>; Limonciello, Mario
> > <Mario_Limonciello@Dell.com>; Pali Rohár <pali.rohar@gmail.com>; Andy
> > Shevchenko <andriy.shevchenko@linux.intel.com>; Rafael Wysocki
> > <rjw@rjwysocki.net>; Andy Lutomirski <luto@amacapital.net>; LKML <linux-
> > kernel@vger.kernel.org>; platform-driver-x86@vger.kernel.org
> > Subject: Re: WMI and Kernel:User interface
> > 
> > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > Linus and Greg,
> > > >
> > > > We are in the process of redesigning the Windows Management
> > Instrumentation
> > > > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-
> > Based
> > > > Enterprise Management (WBEM). We are looking to provide WMI access to
> > userspace,
> > > > while allowing the kernel to filter requests that conflict with its own usage.
> > > > We'd like your take on how this approach relates to our commitment to not
> > break
> > > > userspace.
> > > >
> > > > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > > > devices, consisting of a GUID and a set of methods and events, as well as a
> > > > precompiled intermediate description of the methods and arguments (MOF).
> > Exposed
> > > > to userspace, these methods provide for BIOS interaction and are used for
> > system
> > > > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > > > interest in achieving feature parity with Windows by exposing WMI methods to
> > > > userspace for system management.
> > > >
> > > > While it appears WMI intended to be accessed from userspace, we have
> > > > made use of it in the kernel to support various laptop features by connecting
> > > > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > > > challenge is continuing to use WMI for these platform features, while allowing
> > > > userspace to use it for system management tasks. Unfortunately, the WMI
> > methods
> > > > are not guaranteed to be split up along granular functional lines, and we will
> > > > certainly face situations where the same GUID::METHOD_ID will be needed for
> > a
> > > > kernel feature (say LED support) as well as a system management task.
> > > >
> > > > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > > > the request of and in collaboration with a vendor, with the kernel platform
> > > > driver given the opportunity to filter requests. This filtering would need to be
> > > > at the method and argument inspection level, such as checking for specific bits
> > > > in the input buffer, and rejecting the request if they conflict with an in
> > > > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > > > sufficient).
> > > >
> > > > Because the kernel and the platform drivers are under continual development,
> > and
> > > > new systems appear regularly, we will encounter necessary changes to the
> > > > platform driver WMI request filters. These changes could be considered a
> > change
> > > > to the kernel provided WMI interface to userspace. For example, we could
> > > > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > > > later deny the call when we determine it interferes with kernel usage.
> > > >
> > > > In your view, is it acceptable to provide a chardev interface, for example,
> > > > exposing WMI methods to userspace, with the understanding that the kernel
> > may
> > > > choose to filter certain requests which conflict with its own use? And that this
> > > > filtering may change as new features are added to the platform drivers?
> > >
> > > So, for example, if a new driver for a "brightness key" were added to
> > > the kernel, all of a sudden the "raw" access to the wmi data through the
> > > chardev would filtered away by the kernel and not seen by userspace?
> > >
> > 
> > Pali provided some detail in the rather lengthy thread I linked to [3], but the
> > summary is that there is nothing to encourage vendors to separate out
> > functionality into separate method ids. Some do, the asus-wmi driver seems to
> > have a reasonable separation, others do a lot with a single method id.
> > 
> > In the scenario you mention, it could be that the brightness key shows up in a
> > new laptop. Because we've never seen it before, the kernel driver doesn't detect
> > it and send the input event. The clever user realizes they can write a userspace
> > program to deal with this [a] by writing a my-laptop-brightness-key-daemon which
> > also makes a WMI method call to affect the change in brightness in response to
> > the keypress.
> > 
> > Another user notices the dmesg is reporting "Unknown WMI Event" when they
> > press
> > that key and send a patch to make the corresponding wmi call to affect the
> > brightness change (which depends on driver data to track the brightness value).
> > To avoid conflicting with userspace, they also blacklist the WMI_BRIGHTNESS_CMD
> > from being called by userspace.
> > 
> > Now the first user makes a call to set the brightness, and receives an error
> > code instead of a successful response. Their program no longer works - although
> > with a current kernel, their brightness key would.
> > 
> > I glossed over a few things there, but the description isn't too far off from a
> > plausible scenario.
> > 
> > > Why would you want to do that?  What's wrong with providing "raw" access
> > > through a chardev, and the current in-kernel access as well at the same
> > > time?
> > >
> > > I don't really understand what would "break" over time here.
> > 
> > Like the scenario above, if the behavior has state, if userspace and the kernel
> > attempt to control it, neither will have an accurate state representation.
> > 
> > We could allow both and if things start not working, the user would have to
> > choose between the kernel driver and the userspace management application. This
> > was the scenario Pali was particularly keen to avoid.
> > 
> > That said, to date I haven't come across anything that states there can only be
> > one userspace application accessing WMI at any given time. It should be
> > straightforward to serialize WMI calls such that they cannot "cross the
> > streams".
> > 
> > >
> > > thanks,
> > >
> > > greg k-h
> > >
> > 
> > a. This is perhaps a contrived scenario since hotkeys generate events, and as
> > far as I understand it, there is no interest in exporting events to userspace,
> > just the methods.
> > 
> > --
> > Darren Hart
> > VMware Open Source Technology Center
> 
> This above discussion is confusing because it's referring specifically to "WMI events".
> related to brightness keypresses that don't make sense to go to userspace.
> 
> The more interesting (and potentially problematic) area is things that read and write
> data using ASL methods.
> 
> So here's a "more" realistic scenario:
> 
> OEM has support through a WMI function to control keyboard backlight timeouts
> and intensity.  That same WMI function also can support turning on/off an individual
> USB port.  Backlight timeouts are done by setting the first argument to "1" and USB
> port control is done by setting first argument to "2".
> 
> Some userspace app is developed that can control both of these functions through
> the chardev.  Later an enterprising young kernel developer realizes that backlight 
> control should be done through a platform driver instead.
> 
> They write a platform driver to do it, and add a filter to block "1" arguments from
> userspace.  Now if the userspace app tries to call the chardev with the "1" argument
> some error code is returned indicating this request is not supported.
> 
> The result is the userspace app broke, but it broke because the kernel is supporting
> the method in a much smarter and more scalable way.
> 

Thank you Mario, that's a much better example.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Andy Lutomirski]

On Wed, May 10, 2017 at 3:02 PM,  <Mario.Limonciello@dell.com> wrote:
> So here's a "more" realistic scenario:
>
> OEM has support through a WMI function to control keyboard backlight timeouts
> and intensity.  That same WMI function also can support turning on/off an individual
> USB port.  Backlight timeouts are done by setting the first argument to "1" and USB
> port control is done by setting first argument to "2".
>
> Some userspace app is developed that can control both of these functions through
> the chardev.  Later an enterprising young kernel developer realizes that backlight
> control should be done through a platform driver instead.
>
> They write a platform driver to do it, and add a filter to block "1" arguments from
> userspace.  Now if the userspace app tries to call the chardev with the "1" argument
> some error code is returned indicating this request is not supported.
>
> The result is the userspace app broke, but it broke because the kernel is supporting
> the method in a much smarter and more scalable way.

There's another possibility: the filter could intercept the "1"
argument and change the brightness.

FWIW, I think that almost anything the kernel did with a real WMI API
would be better than the ugly things that drivers like dcdbas allow.

--Andy

Re: WMI and Kernel:User interface

[Darren Hart]

On Wed, May 10, 2017 at 03:50:44PM -0700, Andy Lutomirski wrote:
> On Wed, May 10, 2017 at 3:02 PM,  <Mario.Limonciello@dell.com> wrote:
> > So here's a "more" realistic scenario:
> >
> > OEM has support through a WMI function to control keyboard backlight timeouts
> > and intensity.  That same WMI function also can support turning on/off an individual
> > USB port.  Backlight timeouts are done by setting the first argument to "1" and USB
> > port control is done by setting first argument to "2".
> >
> > Some userspace app is developed that can control both of these functions through
> > the chardev.  Later an enterprising young kernel developer realizes that backlight
> > control should be done through a platform driver instead.
> >
> > They write a platform driver to do it, and add a filter to block "1" arguments from
> > userspace.  Now if the userspace app tries to call the chardev with the "1" argument
> > some error code is returned indicating this request is not supported.
> >
> > The result is the userspace app broke, but it broke because the kernel is supporting
> > the method in a much smarter and more scalable way.
> 
> There's another possibility: the filter could intercept the "1"
> argument and change the brightness.

That would be a good way to deal with regressions if we encounter them after
filtering a call. In general we would want to filter as few calls as possible,
and only if there is a detrimental effect for both userspace and the kernel
using the call. e.g. if the backlight call is stateless, then there is no
problem with both userspace and the kernel using it.

> FWIW, I think that almost anything the kernel did with a real WMI API
> would be better than the ugly things that drivers like dcdbas allow.
> 
> --Andy
> 

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Darren Hart]

On Wed, May 10, 2017 at 10:02:46PM +0000, Mario.Limonciello@dell.com wrote:
> This above discussion is confusing because it's referring specifically to "WMI events".
> related to brightness keypresses that don't make sense to go to userspace.

Let's elaborate on this point. Why are events not interesting to user space?

I don't think it would be too problematic to create a chardev for an Event GUID,
give platform drivers first chance to handle an event, and pass it up to
userspace for exported GUIDs if the kernel doesn't consume the event.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Darren Hart]

On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > Linus and Greg,
> > 
> > We are in the process of redesigning the Windows Management Instrumentation
> > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
> > Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
> > while allowing the kernel to filter requests that conflict with its own usage.
> > We'd like your take on how this approach relates to our commitment to not break
> > userspace.
> > 
> > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > devices, consisting of a GUID and a set of methods and events, as well as a
> > precompiled intermediate description of the methods and arguments (MOF). Exposed
> > to userspace, these methods provide for BIOS interaction and are used for system
> > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > interest in achieving feature parity with Windows by exposing WMI methods to
> > userspace for system management.
> > 
> > While it appears WMI intended to be accessed from userspace, we have
> > made use of it in the kernel to support various laptop features by connecting
> > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > challenge is continuing to use WMI for these platform features, while allowing
> > userspace to use it for system management tasks. Unfortunately, the WMI methods
> > are not guaranteed to be split up along granular functional lines, and we will
> > certainly face situations where the same GUID::METHOD_ID will be needed for a
> > kernel feature (say LED support) as well as a system management task.
> > 
> > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > the request of and in collaboration with a vendor, with the kernel platform
> > driver given the opportunity to filter requests. This filtering would need to be
> > at the method and argument inspection level, such as checking for specific bits
> > in the input buffer, and rejecting the request if they conflict with an in
> > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > sufficient).
> > 
> > Because the kernel and the platform drivers are under continual development, and
> > new systems appear regularly, we will encounter necessary changes to the
> > platform driver WMI request filters. These changes could be considered a change
> > to the kernel provided WMI interface to userspace. For example, we could
> > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > later deny the call when we determine it interferes with kernel usage.
> > 
> > In your view, is it acceptable to provide a chardev interface, for example,
> > exposing WMI methods to userspace, with the understanding that the kernel may
> > choose to filter certain requests which conflict with its own use? And that this
> > filtering may change as new features are added to the platform drivers?
> 
> So, for example, if a new driver for a "brightness key" were added to
> the kernel, all of a sudden the "raw" access to the wmi data through the
> chardev would filtered away by the kernel and not seen by userspace?
> 
> Why would you want to do that?  What's wrong with providing "raw" access
> through a chardev, and the current in-kernel access as well at the same
> time?
> 
> I don't really understand what would "break" over time here.
> 

Just a bump now that we're out of the merge window in case either Greg or Linus
care to follow up with the responses to this.

To Greg's last point - any kernel state that is built up in conjunction with the
WMI interface could be invalidated by a userspace application. It may or may not
be recoverable, depending on the WMI implementation. This would be true for
multiple WMI userspace applications as well, and I suppose the question is, do
we defend the kernel drivers against this, or do we consider the kernel drivers
on equal footing with WMI applications, and say "don't do that then" when some
combination of apps and drivers don't play well together?

> thanks,
> 
> greg k-h
> 

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > Linus and Greg,
> > > 
> > > We are in the process of redesigning the Windows Management Instrumentation
> > > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
> > > Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
> > > while allowing the kernel to filter requests that conflict with its own usage.
> > > We'd like your take on how this approach relates to our commitment to not break
> > > userspace.
> > > 
> > > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > > devices, consisting of a GUID and a set of methods and events, as well as a
> > > precompiled intermediate description of the methods and arguments (MOF). Exposed
> > > to userspace, these methods provide for BIOS interaction and are used for system
> > > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > > interest in achieving feature parity with Windows by exposing WMI methods to
> > > userspace for system management.
> > > 
> > > While it appears WMI intended to be accessed from userspace, we have
> > > made use of it in the kernel to support various laptop features by connecting
> > > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > > challenge is continuing to use WMI for these platform features, while allowing
> > > userspace to use it for system management tasks. Unfortunately, the WMI methods
> > > are not guaranteed to be split up along granular functional lines, and we will
> > > certainly face situations where the same GUID::METHOD_ID will be needed for a
> > > kernel feature (say LED support) as well as a system management task.
> > > 
> > > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > > the request of and in collaboration with a vendor, with the kernel platform
> > > driver given the opportunity to filter requests. This filtering would need to be
> > > at the method and argument inspection level, such as checking for specific bits
> > > in the input buffer, and rejecting the request if they conflict with an in
> > > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > > sufficient).
> > > 
> > > Because the kernel and the platform drivers are under continual development, and
> > > new systems appear regularly, we will encounter necessary changes to the
> > > platform driver WMI request filters. These changes could be considered a change
> > > to the kernel provided WMI interface to userspace. For example, we could
> > > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > > later deny the call when we determine it interferes with kernel usage.
> > > 
> > > In your view, is it acceptable to provide a chardev interface, for example,
> > > exposing WMI methods to userspace, with the understanding that the kernel may
> > > choose to filter certain requests which conflict with its own use? And that this
> > > filtering may change as new features are added to the platform drivers?
> > 
> > So, for example, if a new driver for a "brightness key" were added to
> > the kernel, all of a sudden the "raw" access to the wmi data through the
> > chardev would filtered away by the kernel and not seen by userspace?
> > 
> > Why would you want to do that?  What's wrong with providing "raw" access
> > through a chardev, and the current in-kernel access as well at the same
> > time?
> > 
> > I don't really understand what would "break" over time here.
> > 
> 
> Just a bump now that we're out of the merge window in case either Greg or Linus
> care to follow up with the responses to this.
> 
> To Greg's last point - any kernel state that is built up in conjunction with the
> WMI interface could be invalidated by a userspace application. It may or may not
> be recoverable, depending on the WMI implementation. This would be true for
> multiple WMI userspace applications as well, and I suppose the question is, do
> we defend the kernel drivers against this, or do we consider the kernel drivers
> on equal footing with WMI applications, and say "don't do that then" when some
> combination of apps and drivers don't play well together?

In the end, this shouldn't really matter, as long as nothing breaks as
far as a user notices.  And that's the key here, apis can change, but if
you do it in a way that breaks something, or anyone notices, then it's
not ok.

So I don't have a solid answer other than "good luck!" :)

greg k-h

Re: WMI and Kernel:User interface

[Darren Hart]

On Fri, Jun 09, 2017 at 08:41:51AM +0200, Greg Kroah-Hartman wrote:
> On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman wrote:
> > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > Linus and Greg,
> > > > 
> > > > We are in the process of redesigning the Windows Management Instrumentation
> > > > (WMI) [1] system in the kernel. WMI is the Microsoft implementation of Web-Based
> > > > Enterprise Management (WBEM). We are looking to provide WMI access to userspace,
> > > > while allowing the kernel to filter requests that conflict with its own usage.
> > > > We'd like your take on how this approach relates to our commitment to not break
> > > > userspace.
> > > > 
> > > > For this discussion, we are specifically referring to ACPI PNP0C14 WMI
> > > > devices, consisting of a GUID and a set of methods and events, as well as a
> > > > precompiled intermediate description of the methods and arguments (MOF). Exposed
> > > > to userspace, these methods provide for BIOS interaction and are used for system
> > > > management as well as LEDs, hot keys, radio switches, etc. There is vendor
> > > > interest in achieving feature parity with Windows by exposing WMI methods to
> > > > userspace for system management.
> > > > 
> > > > While it appears WMI intended to be accessed from userspace, we have
> > > > made use of it in the kernel to support various laptop features by connecting
> > > > the WMI methods to other subsystems, notably input, leds, and rfkill [2]. The
> > > > challenge is continuing to use WMI for these platform features, while allowing
> > > > userspace to use it for system management tasks. Unfortunately, the WMI methods
> > > > are not guaranteed to be split up along granular functional lines, and we will
> > > > certainly face situations where the same GUID::METHOD_ID will be needed for a
> > > > kernel feature (say LED support) as well as a system management task.
> > > > 
> > > > To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> > > > the request of and in collaboration with a vendor, with the kernel platform
> > > > driver given the opportunity to filter requests. This filtering would need to be
> > > > at the method and argument inspection level, such as checking for specific bits
> > > > in the input buffer, and rejecting the request if they conflict with an in
> > > > kernel usage (that's worst case, in some cases just GUID or method ID could be
> > > > sufficient).
> > > > 
> > > > Because the kernel and the platform drivers are under continual development, and
> > > > new systems appear regularly, we will encounter necessary changes to the
> > > > platform driver WMI request filters. These changes could be considered a change
> > > > to the kernel provided WMI interface to userspace. For example, we could
> > > > regularly accept a call to $GUID::$METHOD_ID with bit 4 of the buffer set, and
> > > > later deny the call when we determine it interferes with kernel usage.
> > > > 
> > > > In your view, is it acceptable to provide a chardev interface, for example,
> > > > exposing WMI methods to userspace, with the understanding that the kernel may
> > > > choose to filter certain requests which conflict with its own use? And that this
> > > > filtering may change as new features are added to the platform drivers?
> > > 
> > > So, for example, if a new driver for a "brightness key" were added to
> > > the kernel, all of a sudden the "raw" access to the wmi data through the
> > > chardev would filtered away by the kernel and not seen by userspace?
> > > 
> > > Why would you want to do that?  What's wrong with providing "raw" access
> > > through a chardev, and the current in-kernel access as well at the same
> > > time?
> > > 
> > > I don't really understand what would "break" over time here.
> > > 
> > 
> > Just a bump now that we're out of the merge window in case either Greg or Linus
> > care to follow up with the responses to this.
> > 
> > To Greg's last point - any kernel state that is built up in conjunction with the
> > WMI interface could be invalidated by a userspace application. It may or may not
> > be recoverable, depending on the WMI implementation. This would be true for
> > multiple WMI userspace applications as well, and I suppose the question is, do
> > we defend the kernel drivers against this, or do we consider the kernel drivers
> > on equal footing with WMI applications, and say "don't do that then" when some
> > combination of apps and drivers don't play well together?
> 
> In the end, this shouldn't really matter, as long as nothing breaks as
> far as a user notices.  And that's the key here, apis can change, but if
> you do it in a way that breaks something, or anyone notices, then it's
> not ok.
> 
> So I don't have a solid answer other than "good luck!" :)
> 
> greg k-h
> 

Fair enough, thanks Greg.

To all involved. I propose we move forward with:

* Design the API
  - character device per WMI device
  - calling convention
  - platform proof of concept

Let's see this in practice. I hope this proves the point that we do not have to
filter these interfaces, since there is no clean way to do this, and the kernel
module is arguably no more valid a user of WMI than a userspace application
(perhaps less so given the intent of the vendor and the mechanism itself).

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Pali Rohár]

[-- Attachment #1: Type: Text/Plain, Size: 7510 bytes --]

On Saturday 10 June 2017 02:46:41 Darren Hart wrote:
> On Fri, Jun 09, 2017 at 08:41:51AM +0200, Greg Kroah-Hartman wrote:
> > On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> > > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman
> > > wrote:
> > > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > > Linus and Greg,
> > > > > 
> > > > > We are in the process of redesigning the Windows Management
> > > > > Instrumentation (WMI) [1] system in the kernel. WMI is the
> > > > > Microsoft implementation of Web-Based Enterprise Management
> > > > > (WBEM). We are looking to provide WMI access to userspace,
> > > > > while allowing the kernel to filter requests that conflict
> > > > > with its own usage. We'd like your take on how this approach
> > > > > relates to our commitment to not break userspace.
> > > > > 
> > > > > For this discussion, we are specifically referring to ACPI
> > > > > PNP0C14 WMI devices, consisting of a GUID and a set of
> > > > > methods and events, as well as a precompiled intermediate
> > > > > description of the methods and arguments (MOF). Exposed to
> > > > > userspace, these methods provide for BIOS interaction and
> > > > > are used for system management as well as LEDs, hot keys,
> > > > > radio switches, etc. There is vendor interest in achieving
> > > > > feature parity with Windows by exposing WMI methods to
> > > > > userspace for system management.
> > > > > 
> > > > > While it appears WMI intended to be accessed from userspace,
> > > > > we have made use of it in the kernel to support various
> > > > > laptop features by connecting the WMI methods to other
> > > > > subsystems, notably input, leds, and rfkill [2]. The
> > > > > challenge is continuing to use WMI for these platform
> > > > > features, while allowing userspace to use it for system
> > > > > management tasks. Unfortunately, the WMI methods are not
> > > > > guaranteed to be split up along granular functional lines,
> > > > > and we will certainly face situations where the same
> > > > > GUID::METHOD_ID will be needed for a kernel feature (say LED
> > > > > support) as well as a system management task.
> > > > > 
> > > > > To address this, I have proposed [3] that exporting WMI be
> > > > > opt-in, only done at the request of and in collaboration
> > > > > with a vendor, with the kernel platform driver given the
> > > > > opportunity to filter requests. This filtering would need to
> > > > > be at the method and argument inspection level, such as
> > > > > checking for specific bits in the input buffer, and
> > > > > rejecting the request if they conflict with an in kernel
> > > > > usage (that's worst case, in some cases just GUID or method
> > > > > ID could be sufficient).
> > > > > 
> > > > > Because the kernel and the platform drivers are under
> > > > > continual development, and new systems appear regularly, we
> > > > > will encounter necessary changes to the platform driver WMI
> > > > > request filters. These changes could be considered a change
> > > > > to the kernel provided WMI interface to userspace. For
> > > > > example, we could regularly accept a call to
> > > > > $GUID::$METHOD_ID with bit 4 of the buffer set, and later
> > > > > deny the call when we determine it interferes with kernel
> > > > > usage.
> > > > > 
> > > > > In your view, is it acceptable to provide a chardev
> > > > > interface, for example, exposing WMI methods to userspace,
> > > > > with the understanding that the kernel may choose to filter
> > > > > certain requests which conflict with its own use? And that
> > > > > this filtering may change as new features are added to the
> > > > > platform drivers?
> > > > 
> > > > So, for example, if a new driver for a "brightness key" were
> > > > added to the kernel, all of a sudden the "raw" access to the
> > > > wmi data through the chardev would filtered away by the kernel
> > > > and not seen by userspace?
> > > > 
> > > > Why would you want to do that?  What's wrong with providing
> > > > "raw" access through a chardev, and the current in-kernel
> > > > access as well at the same time?
> > > > 
> > > > I don't really understand what would "break" over time here.
> > > 
> > > Just a bump now that we're out of the merge window in case either
> > > Greg or Linus care to follow up with the responses to this.
> > > 
> > > To Greg's last point - any kernel state that is built up in
> > > conjunction with the WMI interface could be invalidated by a
> > > userspace application. It may or may not be recoverable,
> > > depending on the WMI implementation. This would be true for
> > > multiple WMI userspace applications as well, and I suppose the
> > > question is, do we defend the kernel drivers against this, or do
> > > we consider the kernel drivers on equal footing with WMI
> > > applications, and say "don't do that then" when some combination
> > > of apps and drivers don't play well together?
> > 
> > In the end, this shouldn't really matter, as long as nothing breaks

I have one objection here:

If two userspace applications start to fight and use one WMI device at 
same time, then it is their (userspace) problem. In any case (how their 
userspace fight finish) it does not affect kernel nor kernel drivers.

But once some "wrong" or "broken" userspace application starts to fight 
with kernel WMI driver and kernel driver depends on internal WMI state, 
then such application can cause problems to that kernel driver.

And we *must* avoid breakage of kernel drivers by just broken userspace 
application. Such thing could be fatal for kernel and could case 
problems in other kernel drivers (which depends on this WMI kernel 
driver).

I think we cannot accept that some userspace application could use some 
race condition in WMI/ACPI firmware to change internal state of kernel 
WMI driver which could cause undefined behaviour of kernel.

> > as far as a user notices.  And that's the key here, apis can
> > change, but if you do it in a way that breaks something, or anyone
> > notices, then it's not ok.
> > 
> > So I don't have a solid answer other than "good luck!" :)
> > 
> > greg k-h
> 
> Fair enough, thanks Greg.
> 
> To all involved. I propose we move forward with:
> 
> * Design the API
>   - character device per WMI device
>   - calling convention
>   - platform proof of concept
> 
> Let's see this in practice. I hope this proves the point that we do
> not have to filter these interfaces, since there is no clean way to
> do this, and the kernel module is arguably no more valid a user of
> WMI than a userspace application (perhaps less so given the intent
> of the vendor and the mechanism itself).

What do you mean that kernel is no more valid user of WMI? We still have 
a couple of WMI drivers and without correct functionality of them, lot 
of notebooks would less usable for ordinary users.

I still think we need to some some filtering. Kernel and kernel modules 
must not be confused by some random userspace application.

Question about API:

Are we going to export low-level ACPI methods? Or are we going to 
implement BMOF parser in kernel and export high-level class/function API 
to userspace with validation of input parameters in kernel (to prevent 
passing garbage from userspace to ACPI-WMI) like it is on Windows?

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: WMI and Kernel:User interface

[Darren Hart]

On Sat, Jun 10, 2017 at 12:36:40PM +0200, Pali Rohár wrote:
> On Saturday 10 June 2017 02:46:41 Darren Hart wrote:
> > On Fri, Jun 09, 2017 at 08:41:51AM +0200, Greg Kroah-Hartman wrote:
> > > On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> > > > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman
> > > > wrote:
> > > > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > > > Linus and Greg,
> > > > > > 
> > > > > > We are in the process of redesigning the Windows Management
> > > > > > Instrumentation (WMI) [1] system in the kernel. WMI is the
> > > > > > Microsoft implementation of Web-Based Enterprise Management
> > > > > > (WBEM). We are looking to provide WMI access to userspace,
> > > > > > while allowing the kernel to filter requests that conflict
> > > > > > with its own usage. We'd like your take on how this approach
> > > > > > relates to our commitment to not break userspace.
> > > > > > 
> > > > > > For this discussion, we are specifically referring to ACPI
> > > > > > PNP0C14 WMI devices, consisting of a GUID and a set of
> > > > > > methods and events, as well as a precompiled intermediate
> > > > > > description of the methods and arguments (MOF). Exposed to
> > > > > > userspace, these methods provide for BIOS interaction and
> > > > > > are used for system management as well as LEDs, hot keys,
> > > > > > radio switches, etc. There is vendor interest in achieving
> > > > > > feature parity with Windows by exposing WMI methods to
> > > > > > userspace for system management.
> > > > > > 
> > > > > > While it appears WMI intended to be accessed from userspace,
> > > > > > we have made use of it in the kernel to support various
> > > > > > laptop features by connecting the WMI methods to other
> > > > > > subsystems, notably input, leds, and rfkill [2]. The
> > > > > > challenge is continuing to use WMI for these platform
> > > > > > features, while allowing userspace to use it for system
> > > > > > management tasks. Unfortunately, the WMI methods are not
> > > > > > guaranteed to be split up along granular functional lines,
> > > > > > and we will certainly face situations where the same
> > > > > > GUID::METHOD_ID will be needed for a kernel feature (say LED
> > > > > > support) as well as a system management task.
> > > > > > 
> > > > > > To address this, I have proposed [3] that exporting WMI be
> > > > > > opt-in, only done at the request of and in collaboration
> > > > > > with a vendor, with the kernel platform driver given the
> > > > > > opportunity to filter requests. This filtering would need to
> > > > > > be at the method and argument inspection level, such as
> > > > > > checking for specific bits in the input buffer, and
> > > > > > rejecting the request if they conflict with an in kernel
> > > > > > usage (that's worst case, in some cases just GUID or method
> > > > > > ID could be sufficient).
> > > > > > 
> > > > > > Because the kernel and the platform drivers are under
> > > > > > continual development, and new systems appear regularly, we
> > > > > > will encounter necessary changes to the platform driver WMI
> > > > > > request filters. These changes could be considered a change
> > > > > > to the kernel provided WMI interface to userspace. For
> > > > > > example, we could regularly accept a call to
> > > > > > $GUID::$METHOD_ID with bit 4 of the buffer set, and later
> > > > > > deny the call when we determine it interferes with kernel
> > > > > > usage.
> > > > > > 
> > > > > > In your view, is it acceptable to provide a chardev
> > > > > > interface, for example, exposing WMI methods to userspace,
> > > > > > with the understanding that the kernel may choose to filter
> > > > > > certain requests which conflict with its own use? And that
> > > > > > this filtering may change as new features are added to the
> > > > > > platform drivers?
> > > > > 
> > > > > So, for example, if a new driver for a "brightness key" were
> > > > > added to the kernel, all of a sudden the "raw" access to the
> > > > > wmi data through the chardev would filtered away by the kernel
> > > > > and not seen by userspace?
> > > > > 
> > > > > Why would you want to do that?  What's wrong with providing
> > > > > "raw" access through a chardev, and the current in-kernel
> > > > > access as well at the same time?
> > > > > 
> > > > > I don't really understand what would "break" over time here.
> > > > 
> > > > Just a bump now that we're out of the merge window in case either
> > > > Greg or Linus care to follow up with the responses to this.
> > > > 
> > > > To Greg's last point - any kernel state that is built up in
> > > > conjunction with the WMI interface could be invalidated by a
> > > > userspace application. It may or may not be recoverable,
> > > > depending on the WMI implementation. This would be true for
> > > > multiple WMI userspace applications as well, and I suppose the
> > > > question is, do we defend the kernel drivers against this, or do
> > > > we consider the kernel drivers on equal footing with WMI
> > > > applications, and say "don't do that then" when some combination
> > > > of apps and drivers don't play well together?
> > > 
> > > In the end, this shouldn't really matter, as long as nothing breaks
> 
> I have one objection here:
> 
> If two userspace applications start to fight and use one WMI device at 
> same time, then it is their (userspace) problem. In any case (how their 
> userspace fight finish) it does not affect kernel nor kernel drivers.
> 
> But once some "wrong" or "broken" userspace application starts to fight 
> with kernel WMI driver and kernel driver depends on internal WMI state, 
> then such application can cause problems to that kernel driver.
> 
> And we *must* avoid breakage of kernel drivers by just broken userspace 
> application. Such thing could be fatal for kernel and could case 
> problems in other kernel drivers (which depends on this WMI kernel 
> driver).
> 
> I think we cannot accept that some userspace application could use some 
> race condition in WMI/ACPI firmware to change internal state of kernel 
> WMI driver which could cause undefined behaviour of kernel.
> 

These are valid concerns in my opinion. They are, in part, of our own making
(see below re userspace daemons and the mapping driver). I believe we are at the
beginning of a fundamental shift in how we make use of WMI in the Linux world.
There will be some rough patches with the pre-existing WMI drivers within the
Linux kernel.

Admittedly, all of the dangerous corner cases are not evident - at least not to
me. This is in part why I proposed we take the first step, and use something
concrete to help expose such things.

> > > as far as a user notices.  And that's the key here, apis can
> > > change, but if you do it in a way that breaks something, or anyone
> > > notices, then it's not ok.
> > > 
> > > So I don't have a solid answer other than "good luck!" :)
> > > 
> > > greg k-h
> > 
> > Fair enough, thanks Greg.
> > 
> > To all involved. I propose we move forward with:
> > 
> > * Design the API
> >   - character device per WMI device
> >   - calling convention
> >   - platform proof of concept
> > 
> > Let's see this in practice. I hope this proves the point that we do
> > not have to filter these interfaces, since there is no clean way to
> > do this, and the kernel module is arguably no more valid a user of
> > WMI than a userspace application (perhaps less so given the intent
> > of the vendor and the mechanism itself).
> 
> What do you mean that kernel is no more valid user of WMI? We still have 
> a couple of WMI drivers and without correct functionality of them, lot 
> of notebooks would less usable for ordinary users.

Perhaps a poor choice of wording on my part. My point was just that for a system
like WMI, which was designed to expose firmware control to userspace, it is
inconsistent with the intent of the mechanism to grant the Linux kernel a
position of higher importance than that of a userspace management application.

Arguably, implementing platform support with WMI through Linux kernel modules
slows platform support as it is hindered by the barrier to entry and the
kernel's release process - while a generic WMI interface to userspace can be
readily used to write userspace drivers for these features. Obviously, there are
still some gaps with interaction with other subsystems.

WMI may prove to be another one of those subsystems where userspace drivers
facilitate faster innovation and platform enabling. This also helps reduce the
dead code we build up in the kernel supporting devices with relatively short
lifespans.

> I still think we need to some some filtering. Kernel and kernel modules 
> must not be confused by some random userspace application.

Keep in mind that WMI access will be a privileged operation, and I don't think
characterizing these as "random userspace applications" is any more valid than
"random kernel modules".

In hindsight, I suspect implementing more than the mapping driver in kernel
space will prove to have been a mistake. I understand why we did it, and why
that precedent expanded as it has, but at speed and scale, the support these
drivers provide would probably have been better implemented as userspace
platform daemons using the WMI interfaces.

> Question about API:
> 
> Are we going to export low-level ACPI methods? Or are we going to 
> implement BMOF parser in kernel and export high-level class/function API 
> to userspace with validation of input parameters in kernel (to prevent 
> passing garbage from userspace to ACPI-WMI) like it is on Windows?

What we have discussed to date is exposing each WMI device as a character
device. Userspace would select the method and pass a buffer formatted per the
information provided by a userspace BMOF parser. This is consistent with the
design goals of WMI, per the documentation. Specifically, that the mapping
driver does not have any specific knowledge about the WMI GUIDs or methods.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Pali Rohár]

[-- Attachment #1: Type: Text/Plain, Size: 13002 bytes --]

On Monday 12 June 2017 19:02:49 Darren Hart wrote:
> On Sat, Jun 10, 2017 at 12:36:40PM +0200, Pali Rohár wrote:
> > On Saturday 10 June 2017 02:46:41 Darren Hart wrote:
> > > On Fri, Jun 09, 2017 at 08:41:51AM +0200, Greg Kroah-Hartman
> > > wrote:
> > > > On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> > > > > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman
> > > > > 
> > > > > wrote:
> > > > > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > > > > Linus and Greg,
> > > > > > > 
> > > > > > > We are in the process of redesigning the Windows
> > > > > > > Management Instrumentation (WMI) [1] system in the
> > > > > > > kernel. WMI is the Microsoft implementation of Web-Based
> > > > > > > Enterprise Management (WBEM). We are looking to provide
> > > > > > > WMI access to userspace, while allowing the kernel to
> > > > > > > filter requests that conflict with its own usage. We'd
> > > > > > > like your take on how this approach relates to our
> > > > > > > commitment to not break userspace.
> > > > > > > 
> > > > > > > For this discussion, we are specifically referring to
> > > > > > > ACPI PNP0C14 WMI devices, consisting of a GUID and a set
> > > > > > > of methods and events, as well as a precompiled
> > > > > > > intermediate description of the methods and arguments
> > > > > > > (MOF). Exposed to userspace, these methods provide for
> > > > > > > BIOS interaction and are used for system management as
> > > > > > > well as LEDs, hot keys, radio switches, etc. There is
> > > > > > > vendor interest in achieving feature parity with Windows
> > > > > > > by exposing WMI methods to userspace for system
> > > > > > > management.
> > > > > > > 
> > > > > > > While it appears WMI intended to be accessed from
> > > > > > > userspace, we have made use of it in the kernel to
> > > > > > > support various laptop features by connecting the WMI
> > > > > > > methods to other subsystems, notably input, leds, and
> > > > > > > rfkill [2]. The challenge is continuing to use WMI for
> > > > > > > these platform features, while allowing userspace to use
> > > > > > > it for system management tasks. Unfortunately, the WMI
> > > > > > > methods are not guaranteed to be split up along granular
> > > > > > > functional lines, and we will certainly face situations
> > > > > > > where the same GUID::METHOD_ID will be needed for a
> > > > > > > kernel feature (say LED support) as well as a system
> > > > > > > management task.
> > > > > > > 
> > > > > > > To address this, I have proposed [3] that exporting WMI
> > > > > > > be opt-in, only done at the request of and in
> > > > > > > collaboration with a vendor, with the kernel platform
> > > > > > > driver given the opportunity to filter requests. This
> > > > > > > filtering would need to be at the method and argument
> > > > > > > inspection level, such as checking for specific bits in
> > > > > > > the input buffer, and rejecting the request if they
> > > > > > > conflict with an in kernel usage (that's worst case, in
> > > > > > > some cases just GUID or method ID could be sufficient).
> > > > > > > 
> > > > > > > Because the kernel and the platform drivers are under
> > > > > > > continual development, and new systems appear regularly,
> > > > > > > we will encounter necessary changes to the platform
> > > > > > > driver WMI request filters. These changes could be
> > > > > > > considered a change to the kernel provided WMI interface
> > > > > > > to userspace. For example, we could regularly accept a
> > > > > > > call to
> > > > > > > $GUID::$METHOD_ID with bit 4 of the buffer set, and later
> > > > > > > deny the call when we determine it interferes with kernel
> > > > > > > usage.
> > > > > > > 
> > > > > > > In your view, is it acceptable to provide a chardev
> > > > > > > interface, for example, exposing WMI methods to
> > > > > > > userspace, with the understanding that the kernel may
> > > > > > > choose to filter certain requests which conflict with
> > > > > > > its own use? And that this filtering may change as new
> > > > > > > features are added to the platform drivers?
> > > > > > 
> > > > > > So, for example, if a new driver for a "brightness key"
> > > > > > were added to the kernel, all of a sudden the "raw" access
> > > > > > to the wmi data through the chardev would filtered away by
> > > > > > the kernel and not seen by userspace?
> > > > > > 
> > > > > > Why would you want to do that?  What's wrong with providing
> > > > > > "raw" access through a chardev, and the current in-kernel
> > > > > > access as well at the same time?
> > > > > > 
> > > > > > I don't really understand what would "break" over time
> > > > > > here.
> > > > > 
> > > > > Just a bump now that we're out of the merge window in case
> > > > > either Greg or Linus care to follow up with the responses to
> > > > > this.
> > > > > 
> > > > > To Greg's last point - any kernel state that is built up in
> > > > > conjunction with the WMI interface could be invalidated by a
> > > > > userspace application. It may or may not be recoverable,
> > > > > depending on the WMI implementation. This would be true for
> > > > > multiple WMI userspace applications as well, and I suppose
> > > > > the question is, do we defend the kernel drivers against
> > > > > this, or do we consider the kernel drivers on equal footing
> > > > > with WMI applications, and say "don't do that then" when
> > > > > some combination of apps and drivers don't play well
> > > > > together?
> > > > 
> > > > In the end, this shouldn't really matter, as long as nothing
> > > > breaks
> > 
> > I have one objection here:
> > 
> > If two userspace applications start to fight and use one WMI device
> > at same time, then it is their (userspace) problem. In any case
> > (how their userspace fight finish) it does not affect kernel nor
> > kernel drivers.
> > 
> > But once some "wrong" or "broken" userspace application starts to
> > fight with kernel WMI driver and kernel driver depends on internal
> > WMI state, then such application can cause problems to that kernel
> > driver.
> > 
> > And we *must* avoid breakage of kernel drivers by just broken
> > userspace application. Such thing could be fatal for kernel and
> > could case problems in other kernel drivers (which depends on this
> > WMI kernel driver).
> > 
> > I think we cannot accept that some userspace application could use
> > some race condition in WMI/ACPI firmware to change internal state
> > of kernel WMI driver which could cause undefined behaviour of
> > kernel.
> 
> These are valid concerns in my opinion. They are, in part, of our own
> making (see below re userspace daemons and the mapping driver). I
> believe we are at the beginning of a fundamental shift in how we
> make use of WMI in the Linux world. There will be some rough patches
> with the pre-existing WMI drivers within the Linux kernel.
> 
> Admittedly, all of the dangerous corner cases are not evident - at
> least not to me. This is in part why I proposed we take the first
> step, and use something concrete to help expose such things.
> 
> > > > as far as a user notices.  And that's the key here, apis can
> > > > change, but if you do it in a way that breaks something, or
> > > > anyone notices, then it's not ok.
> > > > 
> > > > So I don't have a solid answer other than "good luck!" :)
> > > > 
> > > > greg k-h
> > > 
> > > Fair enough, thanks Greg.
> > > 
> > > To all involved. I propose we move forward with:
> > > 
> > > * Design the API
> > > 
> > >   - character device per WMI device
> > >   - calling convention
> > >   - platform proof of concept
> > > 
> > > Let's see this in practice. I hope this proves the point that we
> > > do not have to filter these interfaces, since there is no clean
> > > way to do this, and the kernel module is arguably no more valid
> > > a user of WMI than a userspace application (perhaps less so
> > > given the intent of the vendor and the mechanism itself).
> > 
> > What do you mean that kernel is no more valid user of WMI? We still
> > have a couple of WMI drivers and without correct functionality of
> > them, lot of notebooks would less usable for ordinary users.
> 
> Perhaps a poor choice of wording on my part. My point was just that
> for a system like WMI, which was designed to expose firmware control
> to userspace, it is inconsistent with the intent of the mechanism to
> grant the Linux kernel a position of higher importance than that of
> a userspace management application.

One thing is design of some system, another thing is real usage. We know 
that WMI is used for reporting events when some keys are pressed (e.g. 
Fn keys) or used for enabling/disabling of radio devices or controlling 
keyboard/LID backlight. All those parts are in Linux world handled by 
kernel (not userspace) and therefore WMI needs to be handled in Linux 
kernel.

If other operating systems implements above parts of functionality in 
userspace and based on this fact was WMI designed, then it is not fully 
relevant argument for Linux.

And if we need to ensure correctly functionality of kernel drivers, then 
I think Linux kernel has a position with higher importance as some 
userspace application.

But it does not mean we cannot or do not want export other functionality 
to userspace. Just I still think we should be careful when some 3rd 
party firmware code which running in kernel mode (=ACPI-WMI bytecode) is 
going to be accessed by more users (both kernel drivers and userspace 
applications).

And filter layer which will accept only WMI calls which are safe for 
currently loaded/used kernel modules seems like a sane idea to ensure 
functionality of kernel plus allow userspace to do other things.

> Arguably, implementing platform support with WMI through Linux kernel
> modules slows platform support as it is hindered by the barrier to
> entry and the kernel's release process - while a generic WMI

But we can say mainline kernel has better code quality as some random 
closed source 3rd vendor application.

If we look in Microsoft Windows world (where that WMI is used a lot), we 
have there for function X specific application from each notebook 
vendor. And all those applications which doing function X are 
incompatible and every one locked for particular notebook model.

So I'm not sure if this is right way.

> interface to userspace can be readily used to write userspace
> drivers for these features. Obviously, there are still some gaps
> with interaction with other subsystems.
> 
> WMI may prove to be another one of those subsystems where userspace
> drivers facilitate faster innovation and platform enabling. This
> also helps reduce the dead code we build up in the kernel supporting
> devices with relatively short lifespans.
> 
> > I still think we need to some some filtering. Kernel and kernel
> > modules must not be confused by some random userspace application.
> 
> Keep in mind that WMI access will be a privileged operation, and I
> don't think characterizing these as "random userspace applications"
> is any more valid than "random kernel modules".

Kernel modules (which do not taint kernel) comes with kernel itself. 
They are well known and basically are part of kernel code.

Userspace applications, and if we are talking about WMI, would be 
probably 3rd party vendor closed-source binaries compiled for one or two 
specific Linux distributions. For me it is just "random userspace 
application" which I do not thing that would be preinstalled or part of 
Linux distributions, like it is for coreutils or X Server today.

> In hindsight, I suspect implementing more than the mapping driver in
> kernel space will prove to have been a mistake. I understand why we
> did it, and why that precedent expanded as it has, but at speed and
> scale, the support these drivers provide would probably have been
> better implemented as userspace platform daemons using the WMI
> interfaces.
> 
> > Question about API:
> > 
> > Are we going to export low-level ACPI methods? Or are we going to
> > implement BMOF parser in kernel and export high-level
> > class/function API to userspace with validation of input
> > parameters in kernel (to prevent passing garbage from userspace to
> > ACPI-WMI) like it is on Windows?
> 
> What we have discussed to date is exposing each WMI device as a
> character device. Userspace would select the method and pass a
> buffer formatted per the information provided by a userspace BMOF
> parser. This is consistent with the design goals of WMI, per the
> documentation. Specifically, that the mapping driver does not have
> any specific knowledge about the WMI GUIDs or methods.

Ok.

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 12:17:28AM +0200, Pali Rohár wrote:
> On Monday 12 June 2017 19:02:49 Darren Hart wrote:
> > On Sat, Jun 10, 2017 at 12:36:40PM +0200, Pali Rohár wrote:
> > > On Saturday 10 June 2017 02:46:41 Darren Hart wrote:
> > > > On Fri, Jun 09, 2017 at 08:41:51AM +0200, Greg Kroah-Hartman
> > > > wrote:
> > > > > On Sat, Jun 03, 2017 at 12:50:58PM -0700, Darren Hart wrote:
> > > > > > On Wed, May 10, 2017 at 07:13:41AM +0200, Greg Kroah-Hartman
> > > > > > 
> > > > > > wrote:
> > > > > > > On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:
> > > > > > > > Linus and Greg,
> > > > > > > > 
> > > > > > > > We are in the process of redesigning the Windows
> > > > > > > > Management Instrumentation (WMI) [1] system in the
> > > > > > > > kernel. WMI is the Microsoft implementation of Web-Based
> > > > > > > > Enterprise Management (WBEM). We are looking to provide
> > > > > > > > WMI access to userspace, while allowing the kernel to
> > > > > > > > filter requests that conflict with its own usage. We'd
> > > > > > > > like your take on how this approach relates to our
> > > > > > > > commitment to not break userspace.
> > > > > > > > 
> > > > > > > > For this discussion, we are specifically referring to
> > > > > > > > ACPI PNP0C14 WMI devices, consisting of a GUID and a set
> > > > > > > > of methods and events, as well as a precompiled
> > > > > > > > intermediate description of the methods and arguments
> > > > > > > > (MOF). Exposed to userspace, these methods provide for
> > > > > > > > BIOS interaction and are used for system management as
> > > > > > > > well as LEDs, hot keys, radio switches, etc. There is
> > > > > > > > vendor interest in achieving feature parity with Windows
> > > > > > > > by exposing WMI methods to userspace for system
> > > > > > > > management.
> > > > > > > > 
> > > > > > > > While it appears WMI intended to be accessed from
> > > > > > > > userspace, we have made use of it in the kernel to
> > > > > > > > support various laptop features by connecting the WMI
> > > > > > > > methods to other subsystems, notably input, leds, and
> > > > > > > > rfkill [2]. The challenge is continuing to use WMI for
> > > > > > > > these platform features, while allowing userspace to use
> > > > > > > > it for system management tasks. Unfortunately, the WMI
> > > > > > > > methods are not guaranteed to be split up along granular
> > > > > > > > functional lines, and we will certainly face situations
> > > > > > > > where the same GUID::METHOD_ID will be needed for a
> > > > > > > > kernel feature (say LED support) as well as a system
> > > > > > > > management task.
> > > > > > > > 
> > > > > > > > To address this, I have proposed [3] that exporting WMI
> > > > > > > > be opt-in, only done at the request of and in
> > > > > > > > collaboration with a vendor, with the kernel platform
> > > > > > > > driver given the opportunity to filter requests. This
> > > > > > > > filtering would need to be at the method and argument
> > > > > > > > inspection level, such as checking for specific bits in
> > > > > > > > the input buffer, and rejecting the request if they
> > > > > > > > conflict with an in kernel usage (that's worst case, in
> > > > > > > > some cases just GUID or method ID could be sufficient).
> > > > > > > > 
> > > > > > > > Because the kernel and the platform drivers are under
> > > > > > > > continual development, and new systems appear regularly,
> > > > > > > > we will encounter necessary changes to the platform
> > > > > > > > driver WMI request filters. These changes could be
> > > > > > > > considered a change to the kernel provided WMI interface
> > > > > > > > to userspace. For example, we could regularly accept a
> > > > > > > > call to
> > > > > > > > $GUID::$METHOD_ID with bit 4 of the buffer set, and later
> > > > > > > > deny the call when we determine it interferes with kernel
> > > > > > > > usage.
> > > > > > > > 
> > > > > > > > In your view, is it acceptable to provide a chardev
> > > > > > > > interface, for example, exposing WMI methods to
> > > > > > > > userspace, with the understanding that the kernel may
> > > > > > > > choose to filter certain requests which conflict with
> > > > > > > > its own use? And that this filtering may change as new
> > > > > > > > features are added to the platform drivers?
> > > > > > > 
> > > > > > > So, for example, if a new driver for a "brightness key"
> > > > > > > were added to the kernel, all of a sudden the "raw" access
> > > > > > > to the wmi data through the chardev would filtered away by
> > > > > > > the kernel and not seen by userspace?
> > > > > > > 
> > > > > > > Why would you want to do that?  What's wrong with providing
> > > > > > > "raw" access through a chardev, and the current in-kernel
> > > > > > > access as well at the same time?
> > > > > > > 
> > > > > > > I don't really understand what would "break" over time
> > > > > > > here.
> > > > > > 
> > > > > > Just a bump now that we're out of the merge window in case
> > > > > > either Greg or Linus care to follow up with the responses to
> > > > > > this.
> > > > > > 
> > > > > > To Greg's last point - any kernel state that is built up in
> > > > > > conjunction with the WMI interface could be invalidated by a
> > > > > > userspace application. It may or may not be recoverable,
> > > > > > depending on the WMI implementation. This would be true for
> > > > > > multiple WMI userspace applications as well, and I suppose
> > > > > > the question is, do we defend the kernel drivers against
> > > > > > this, or do we consider the kernel drivers on equal footing
> > > > > > with WMI applications, and say "don't do that then" when
> > > > > > some combination of apps and drivers don't play well
> > > > > > together?
> > > > > 
> > > > > In the end, this shouldn't really matter, as long as nothing
> > > > > breaks
> > > 
> > > I have one objection here:
> > > 
> > > If two userspace applications start to fight and use one WMI device
> > > at same time, then it is their (userspace) problem. In any case
> > > (how their userspace fight finish) it does not affect kernel nor
> > > kernel drivers.
> > > 
> > > But once some "wrong" or "broken" userspace application starts to
> > > fight with kernel WMI driver and kernel driver depends on internal
> > > WMI state, then such application can cause problems to that kernel
> > > driver.
> > > 
> > > And we *must* avoid breakage of kernel drivers by just broken
> > > userspace application. Such thing could be fatal for kernel and
> > > could case problems in other kernel drivers (which depends on this
> > > WMI kernel driver).
> > > 
> > > I think we cannot accept that some userspace application could use
> > > some race condition in WMI/ACPI firmware to change internal state
> > > of kernel WMI driver which could cause undefined behaviour of
> > > kernel.
> > 
> > These are valid concerns in my opinion. They are, in part, of our own
> > making (see below re userspace daemons and the mapping driver). I
> > believe we are at the beginning of a fundamental shift in how we
> > make use of WMI in the Linux world. There will be some rough patches
> > with the pre-existing WMI drivers within the Linux kernel.
> > 
> > Admittedly, all of the dangerous corner cases are not evident - at
> > least not to me. This is in part why I proposed we take the first
> > step, and use something concrete to help expose such things.
> > 
> > > > > as far as a user notices.  And that's the key here, apis can
> > > > > change, but if you do it in a way that breaks something, or
> > > > > anyone notices, then it's not ok.
> > > > > 
> > > > > So I don't have a solid answer other than "good luck!" :)
> > > > > 
> > > > > greg k-h
> > > > 
> > > > Fair enough, thanks Greg.
> > > > 
> > > > To all involved. I propose we move forward with:
> > > > 
> > > > * Design the API
> > > > 
> > > >   - character device per WMI device
> > > >   - calling convention
> > > >   - platform proof of concept
> > > > 
> > > > Let's see this in practice. I hope this proves the point that we
> > > > do not have to filter these interfaces, since there is no clean
> > > > way to do this, and the kernel module is arguably no more valid
> > > > a user of WMI than a userspace application (perhaps less so
> > > > given the intent of the vendor and the mechanism itself).

Hey Pali,

I found it difficult to respond to your points without droning on about some the
positions I've developed over the years regarding open source projects versus
products, platform enabling, and enabling people/companies to innovate
independently. So while I don't necessarily want to debate all the finer points,
I probably do need to get this out there at some point. So I'll do that here,
and then try to bring this back to "OK fine, but are we actually going to do
about it all?" as we continue the discussion.

I think it is accurate to say that you are responding from a "where we are now"
perspective, while much of what I'll say below is from a "what might this look
like ideally?". Ultimately our solution has to start from where we are now, but
I do want us to consider what the ideal might look like.

> > > 
> > > What do you mean that kernel is no more valid user of WMI? We still
> > > have a couple of WMI drivers and without correct functionality of
> > > them, lot of notebooks would less usable for ordinary users.
> > 
> > Perhaps a poor choice of wording on my part. My point was just that
> > for a system like WMI, which was designed to expose firmware control
> > to userspace, it is inconsistent with the intent of the mechanism to
> > grant the Linux kernel a position of higher importance than that of
> > a userspace management application.
> 
> One thing is design of some system, another thing is real usage. We know 
> that WMI is used for reporting events when some keys are pressed (e.g. 
> Fn keys) or used for enabling/disabling of radio devices or controlling 
> keyboard/LID backlight. All those parts are in Linux world handled by 
> kernel (not userspace) and therefore WMI needs to be handled in Linux 
> kernel.

These are examples of the subsystems I was referring to below. We can already
assign LED triggers for some things. However, when the WMI method is
specifically used as the mechanism for toggling the LED... yes, you make a good
point.

> 
> If other operating systems implements above parts of functionality in 
> userspace and based on this fact was WMI designed, then it is not fully 
> relevant argument for Linux.
> 

This is a big topic for sure. Speed and scale of platform enabling is something
I would like to see us support better. The barrier to entry to kernel
changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
which would ideally, IMHO, be in the hands of the OEMs.

To the point above, the OEMs design these systems. For a long time they mostly
just ignored Linux, so we've done what we have to in order to get the last 5% of
the platform working. This approach was done in a way that is efficient for us,
so when an OEM comes in and wants to support their own product on Linux -
they've faced with a system that wasn't designed to be efficient for them.

The design of the system is relevant to the Linux kernel as working against the
intent of the design frequently results in difficult to maintain code, often
reverse engineered and dependent upon undefined behavior, which trips up against
weird corner cases, updates, etc.

Ideally, we would provide a generic way for users/OEMs/vendors to successfully
support and maintain their own platforms, ideally with as little kernel changes
as possible. If we can get closer to that than we are today with this WMI work,
I think that is worth the effort.

> And if we need to ensure correctly functionality of kernel drivers, then 
> I think Linux kernel has a position with higher importance as some 
> userspace application.
> 
> But it does not mean we cannot or do not want export other functionality 
> to userspace. Just I still think we should be careful when some 3rd 
> party firmware code which running in kernel mode (=ACPI-WMI bytecode) is 
> going to be accessed by more users (both kernel drivers and userspace 
> applications).
> 
> And filter layer which will accept only WMI calls which are safe for 
> currently loaded/used kernel modules seems like a sane idea to ensure 
> functionality of kernel plus allow userspace to do other things.

My biggest concern with this approach is maintenance. Because we would be doing
something unforeseen by the specification, the various vendor implemented WMI
APIs are not likely to be amenable to filtering. I can see these filters
getting extremely complicated. They are "high touch", by which I mean each
generation of platform may require subtle tweaks, which will be difficult to
verify they don't break past generations.

> 
> > Arguably, implementing platform support with WMI through Linux kernel
> > modules slows platform support as it is hindered by the barrier to
> > entry and the kernel's release process - while a generic WMI
> 
> But we can say mainline kernel has better code quality as some random 
> closed source 3rd vendor application.

While I think we work pretty hard at writing quality code, the code analysis
studies I'm aware of (such as those being developed by the SIL2Linux MP project
for the use of the Linux kernel in safety critical systems) support this
statement for the most core code of the Linux kernel, the stuff that
everything needs. This doesn't hold true of all parts of the kernel, however,
with lower use drivers showing some of the highest bug rates of all subsystems
(along with certain filesystems).

> If we look in Microsoft Windows world (where that WMI is used a lot), we 
> have there for function X specific application from each notebook 
> vendor. And all those applications which doing function X are 
> incompatible and every one locked for particular notebook model.
> 
> So I'm not sure if this is right way.
> 

There are certainly pros and cons. While this approach results in duplication of
effort, it also allows vendors to "own their own destiny" and innovate and
support their platforms independently. It also minimizes the amount of dead code
accumulating for platforms that just don't exist for very long.

Over the years I've become more and more supportive of the idea that the kernel is the
place for common code everything needs - and specifically not the code that will
live for 18 months and never be needed again. The development and maintenance
models for these two things are fundamentally different.

In general, I want to work to support the development of interfaces that allow
people and companies to develop their new thing, without forcing them to come
back to us repeatedly. "Enable them to enable themselves and get out of their
way."

> > interface to userspace can be readily used to write userspace
> > drivers for these features. Obviously, there are still some gaps
> > with interaction with other subsystems.
> > 
> > WMI may prove to be another one of those subsystems where userspace
> > drivers facilitate faster innovation and platform enabling. This
> > also helps reduce the dead code we build up in the kernel supporting
> > devices with relatively short lifespans.
> > 
> > > I still think we need to some some filtering. Kernel and kernel
> > > modules must not be confused by some random userspace application.
> > 
> > Keep in mind that WMI access will be a privileged operation, and I
> > don't think characterizing these as "random userspace applications"
> > is any more valid than "random kernel modules".
> 
> Kernel modules (which do not taint kernel) comes with kernel itself. 
> They are well known and basically are part of kernel code.

This is a fairly kernel-centric perspective. It works great for us, but
many non-kernel-devs will have a different view.

> 
> Userspace applications, and if we are talking about WMI, would be 
> probably 3rd party vendor closed-source binaries compiled for one or two 
> specific Linux distributions. For me it is just "random userspace 
> application" which I do not thing that would be preinstalled or part of 
> Linux distributions, like it is for coreutils or X Server today.

That is certainly not the outcome we'd be aiming for.

> > In hindsight, I suspect implementing more than the mapping driver in
> > kernel space will prove to have been a mistake. I understand why we
> > did it, and why that precedent expanded as it has, but at speed and
> > scale, the support these drivers provide would probably have been
> > better implemented as userspace platform daemons using the WMI
> > interfaces.
> > 
> > > Question about API:
> > > 
> > > Are we going to export low-level ACPI methods? Or are we going to
> > > implement BMOF parser in kernel and export high-level
> > > class/function API to userspace with validation of input
> > > parameters in kernel (to prevent passing garbage from userspace to
> > > ACPI-WMI) like it is on Windows?
> > 
> > What we have discussed to date is exposing each WMI device as a
> > character device. Userspace would select the method and pass a
> > buffer formatted per the information provided by a userspace BMOF
> > parser. This is consistent with the design goals of WMI, per the
> > documentation. Specifically, that the mapping driver does not have
> > any specific knowledge about the WMI GUIDs or methods.
> 
> Ok.
> 

As to the "OK fine, but what are we actually going to do..." bit...

In order to support broad enabling, we should avoid Whitelists which would
require we add new GUIDs to the kernel as fast as vendors can add them.

If we require filtering, it should be along the lines of an ACCEPT/DENY filter,
which only denies specific accesses. Drivers which match a WMI GUID can register
a filter callback, returning true or false (accept or deny). This would allow
for the creation of management tools in cooperation with the existing
drivers, without precluding the development of userspace platform-daemons which
could handle all aspects of the WMI interface (if they conflict with an existing
driver, they would need to run without that driver, which would avoid loading
the WMI filter).

Possible concerns, as before, we run the risk of a new driver being written, or
a new feature being added to an existing driver, that needs to add a filter
which would then deny userspace accesses previously allowed. It becomes a first
to support the platform race.

I don't see a way to both make WMI broadly useful and protect existing drivers
without this potential risk.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Christoph Hellwig]

Hi Darren,

first - can you please properly trim your replies and don't write
more than 7 characters per line?

On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> This is a big topic for sure. Speed and scale of platform enabling is something
> I would like to see us support better. The barrier to entry to kernel
> changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> which would ideally, IMHO, be in the hands of the OEMs.

It's not.  It's a trivial patch, and you cover all Linux users.  Very
much unlike say the windows world where you are stuck with installing
a vendor specific set of drivers forever.

> Ideally, we would provide a generic way for users/OEMs/vendors to successfully
> support and maintain their own platforms, ideally with as little kernel changes
> as possible. If we can get closer to that than we are today with this WMI work,
> I think that is worth the effort.

Hell no!  The last thing we need on Linux is systems that once support
us added don't just work out of the box because you're missing your
vendor blob.

> > And filter layer which will accept only WMI calls which are safe for 
> > currently loaded/used kernel modules seems like a sane idea to ensure 
> > functionality of kernel plus allow userspace to do other things.
> 
> My biggest concern with this approach is maintenance. Because we would be doing
> something unforeseen by the specification, the various vendor implemented WMI
> APIs are not likely to be amenable to filtering. I can see these filters
> getting extremely complicated. They are "high touch", by which I mean each
> generation of platform may require subtle tweaks, which will be difficult to
> verify they don't break past generations.

Agreed.  As mentioned before I think the only sensible approach is
white listing GUIDs that have a valid userspace use case.  And use
the dynamic IDs approach to add them for debugging and reverse
engineering.

> There are certainly pros and cons. While this approach results in duplication of
> effort, it also allows vendors to "own their own destiny" and innovate and
> support their platforms independently. It also minimizes the amount of dead code
> accumulating for platforms that just don't exist for very long.

Bullshit alert..

Re: WMI and Kernel:User interface

[Pali Rohár]

On Tuesday 13 June 2017 00:05:35 Christoph Hellwig wrote:
> On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> > This is a big topic for sure. Speed and scale of platform enabling is something
> > I would like to see us support better. The barrier to entry to kernel
> > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> > which would ideally, IMHO, be in the hands of the OEMs.
> 
> It's not.  It's a trivial patch, and you cover all Linux users.  Very
> much unlike say the windows world where you are stuck with installing
> a vendor specific set of drivers forever.

Yes, adding new GUID is same hard as adding new PCI ID or USB ID. It is
really trivial patch.

> > Ideally, we would provide a generic way for users/OEMs/vendors to successfully
> > support and maintain their own platforms, ideally with as little kernel changes
> > as possible. If we can get closer to that than we are today with this WMI work,
> > I think that is worth the effort.
> 
> Hell no!  The last thing we need on Linux is systems that once support
> us added don't just work out of the box because you're missing your
> vendor blob.

Seeing vendor blob compiled for one particular userspace and
distribution which would be needed for having working notebook support
on Linux is way to the hell.

Here I agree with Christoph.

> > > And filter layer which will accept only WMI calls which are safe for 
> > > currently loaded/used kernel modules seems like a sane idea to ensure 
> > > functionality of kernel plus allow userspace to do other things.
> > 
> > My biggest concern with this approach is maintenance. Because we would be doing
> > something unforeseen by the specification, the various vendor implemented WMI
> > APIs are not likely to be amenable to filtering. I can see these filters
> > getting extremely complicated. They are "high touch", by which I mean each
> > generation of platform may require subtle tweaks, which will be difficult to
> > verify they don't break past generations.
> 
> Agreed.  As mentioned before I think the only sensible approach is
> white listing GUIDs that have a valid userspace use case.  And use
> the dynamic IDs approach to add them for debugging and reverse
> engineering.

In some cases filter function can be simple in some cases hard. I can
image that usage of while listing, plus in some cases also filtering
(when it would be relatively easy to implement).

-- 
Pali Rohár
pali.rohar@gmail.com

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 02:07:41PM +0200, Pali Rohár wrote:
> On Tuesday 13 June 2017 00:05:35 Christoph Hellwig wrote:
> > On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> > > This is a big topic for sure. Speed and scale of platform enabling is something
> > > I would like to see us support better. The barrier to entry to kernel
> > > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> > > which would ideally, IMHO, be in the hands of the OEMs.
> > 
> > It's not.  It's a trivial patch, and you cover all Linux users.  Very
> > much unlike say the windows world where you are stuck with installing
> > a vendor specific set of drivers forever.
> 
> Yes, adding new GUID is same hard as adding new PCI ID or USB ID. It is
> really trivial patch.


See my response to Christoph - it's not the complexity of the patch, it's the
timeline. As Christoph points out, however, dynamic IDs may address this
concern.

> 
> In some cases filter function can be simple in some cases hard. I can
> image that usage of while listing, plus in some cases also filtering
> (when it would be relatively easy to implement).

See my response to Christoph - to address the concern of breaking userspace
later, if we consider this a proxy instead of a filter, we can make it
transparent to userspace and maintain kernel driver state. The driver can
register a wmi_method_proxy callback which can choose to proxy the method call
or not. If it does, it can update it's own state and perform the requested
action through it's own infrastructure, populate the out buffer and send it back
up to userspace. I would hope to see as few of these as possible, but they would
allow for protecting the kernel drivers while still enabling userspace usage of
WMI.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Tue, Jun 13, 2017 at 08:44:19AM -0700, Darren Hart wrote:
> > In some cases filter function can be simple in some cases hard. I can
> > image that usage of while listing, plus in some cases also filtering
> > (when it would be relatively easy to implement).
> 
> See my response to Christoph - to address the concern of breaking userspace
> later, if we consider this a proxy instead of a filter, we can make it
> transparent to userspace and maintain kernel driver state. The driver can
> register a wmi_method_proxy callback which can choose to proxy the method call
> or not. If it does, it can update it's own state and perform the requested
> action through it's own infrastructure, populate the out buffer and send it back
> up to userspace. I would hope to see as few of these as possible, but they would
> allow for protecting the kernel drivers while still enabling userspace usage of
> WMI.

Ok, I think we need to see some real code here to get any further with
this.  There's a reason we don't do big design discussions without
patches, as that usually answers all of the questions involved.

Please, let's see some code before anything else.

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 06:05:47PM +0200, Greg Kroah-Hartman wrote:
> On Tue, Jun 13, 2017 at 08:44:19AM -0700, Darren Hart wrote:
> > > In some cases filter function can be simple in some cases hard. I can
> > > image that usage of while listing, plus in some cases also filtering
> > > (when it would be relatively easy to implement).
> > 
> > See my response to Christoph - to address the concern of breaking userspace
> > later, if we consider this a proxy instead of a filter, we can make it
> > transparent to userspace and maintain kernel driver state. The driver can
> > register a wmi_method_proxy callback which can choose to proxy the method call
> > or not. If it does, it can update it's own state and perform the requested
> > action through it's own infrastructure, populate the out buffer and send it back
> > up to userspace. I would hope to see as few of these as possible, but they would
> > allow for protecting the kernel drivers while still enabling userspace usage of
> > WMI.
> 
> Ok, I think we need to see some real code here to get any further with
> this.  There's a reason we don't do big design discussions without
> patches, as that usually answers all of the questions involved.
> 
> Please, let's see some code before anything else.

Agreed. That was my intent earlier with getting the chardev and calling
conventions established.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 12:05:35AM -0700, Christoph Hellwig wrote:
> Hi Darren,
> 
> first - can you please properly trim your replies and don't write
> more than 7 characters per line?

Sure... (although I think you've done all the necessary pruning for this
response). 70 I presume you mean? I usually have tw set to 72...
apparently I dropped that setting at some point. Will correct.

> 
> On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> > This is a big topic for sure. Speed and scale of platform enabling is something
> > I would like to see us support better. The barrier to entry to kernel
> > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> > which would ideally, IMHO, be in the hands of the OEMs.
> 
> It's not.  It's a trivial patch, and you cover all Linux users.  Very
> much unlike say the windows world where you are stuck with installing
> a vendor specific set of drivers forever.
> 

The patch is trivial, but the process is time consuming. Two to Three
months to see an ID added and released is big blocker for contemporary
life cycles.

> > Ideally, we would provide a generic way for users/OEMs/vendors to successfully
> > support and maintain their own platforms, ideally with as little kernel changes
> > as possible. If we can get closer to that than we are today with this WMI work,
> > I think that is worth the effort.
> 
> Hell no!  The last thing we need on Linux is systems that once support
> us added don't just work out of the box because you're missing your
> vendor blob.

I thought more about this overnight and changed my thinking a bit. While
I still stand by the position that we should be making it easier for
users/OEMs/vendors to support their platforms (note that I had included
users in there, and I'm specifically referring to many of the people
reporting bugs on laptops who would be more likely to fix the issue if
it could be done outside of the kernel).

I'll mention this again I suspect in this thread, but rather than a "WMI
filter" we can implement a "WMI proxy". If a kernel driver needs to own
certain WMI calls for LED or Radio management, for example, all such
calls can be proxied through that driver. It can do the necessary work
to update its own state, and still perform the requested funtion,
transparent to the userspace caller. This should accommodate the
addition of new drivers and features to kernel drivers, without
precluding the development of userspace management or platform daemons.

> > > And filter layer which will accept only WMI calls which are safe for 
> > > currently loaded/used kernel modules seems like a sane idea to ensure 
> > > functionality of kernel plus allow userspace to do other things.
> > 
> > My biggest concern with this approach is maintenance. Because we would be doing
> > something unforeseen by the specification, the various vendor implemented WMI
> > APIs are not likely to be amenable to filtering. I can see these filters
> > getting extremely complicated. They are "high touch", by which I mean each
> > generation of platform may require subtle tweaks, which will be difficult to
> > verify they don't break past generations.
> 
> Agreed.  As mentioned before I think the only sensible approach is
> white listing GUIDs that have a valid userspace use case.  And use
> the dynamic IDs approach to add them for debugging and reverse
> engineering.

The issue with whitelisting GUIDs is that, for the same reasons as
above, they are not going to be nicely partitioned into functional
chunks that make sense from a kernel perspective. They aren't going to
see it through a subsystems lense, "LEDs, Radios, Hotkeys, BIOS
Management". Much like the fujitsu ACPI devices have nonsensical
interdependencies, these GUIDs and the methods they contain are not
granular enough for filtering.

As to dynamic IDs... to make sure I'm thinking the same thing you are...
you are referring to passing device IDs from userspace, through sysfs,
to a driver at runtime to allow it to bind to an ID it doesn't already
know about? If I have this right, it also addresses my concern above for
adding new IDs taking too long as this can provide an intermediate
solution.

> 
> > There are certainly pros and cons. While this approach results in duplication of
> > effort, it also allows vendors to "own their own destiny" and innovate and
> > support their platforms independently. It also minimizes the amount of dead code
> > accumulating for platforms that just don't exist for very long.
> 
> Bullshit alert..
> 

I probably need to add "innovate" to my bad words dictionary as it gets
this knee jerk response. But what specifically do you object to?
Supporting vendors in product development? Or not accumulating dead code
in the kernel?

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Tue, Jun 13, 2017 at 08:38:57AM -0700, Darren Hart wrote:
> On Tue, Jun 13, 2017 at 12:05:35AM -0700, Christoph Hellwig wrote:
> > Hi Darren,
> > 
> > first - can you please properly trim your replies and don't write
> > more than 7 characters per line?
> 
> Sure... (although I think you've done all the necessary pruning for this
> response). 70 I presume you mean? I usually have tw set to 72...
> apparently I dropped that setting at some point. Will correct.
> 
> > 
> > On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> > > This is a big topic for sure. Speed and scale of platform enabling is something
> > > I would like to see us support better. The barrier to entry to kernel
> > > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> > > which would ideally, IMHO, be in the hands of the OEMs.
> > 
> > It's not.  It's a trivial patch, and you cover all Linux users.  Very
> > much unlike say the windows world where you are stuck with installing
> > a vendor specific set of drivers forever.
> > 
> 
> The patch is trivial, but the process is time consuming. Two to Three
> months to see an ID added and released is big blocker for contemporary
> life cycles.

Wait, what?  Please explain.

Yes, it could take worse case 2-3 months to add a new device id, but
does it really?  I take new device ids up until 2 weeks before a -final
kernel is released.  And once they are in Linus's tree it's usually only
a single week before they end up in all stable kernel releases.

But that's upstream, no device ships with upstream, they ship a distro
kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
device id into their kernels takes what, a day or two?  And that is what
really matters as that is what goes out the door for their device.

At least that is the process for when _I_ used to work on pre-installed
Linux on devices, maybe things have gotten a lot worse since I left that
business, but I would sure hope it wouldn't get magnitudes worse.

So 2-3 months seems really long to me.

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Andy Lutomirski]

On Tue, Jun 13, 2017 at 8:50 AM, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
> On Tue, Jun 13, 2017 at 08:38:57AM -0700, Darren Hart wrote:
>> On Tue, Jun 13, 2017 at 12:05:35AM -0700, Christoph Hellwig wrote:
>> > Hi Darren,
>> >
>> > first - can you please properly trim your replies and don't write
>> > more than 7 characters per line?
>>
>> Sure... (although I think you've done all the necessary pruning for this
>> response). 70 I presume you mean? I usually have tw set to 72...
>> apparently I dropped that setting at some point. Will correct.
>>
>> >
>> > On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
>> > > This is a big topic for sure. Speed and scale of platform enabling is something
>> > > I would like to see us support better. The barrier to entry to kernel
>> > > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
>> > > which would ideally, IMHO, be in the hands of the OEMs.
>> >
>> > It's not.  It's a trivial patch, and you cover all Linux users.  Very
>> > much unlike say the windows world where you are stuck with installing
>> > a vendor specific set of drivers forever.
>> >
>>
>> The patch is trivial, but the process is time consuming. Two to Three
>> months to see an ID added and released is big blocker for contemporary
>> life cycles.
>
> Wait, what?  Please explain.
>
> Yes, it could take worse case 2-3 months to add a new device id, but
> does it really?  I take new device ids up until 2 weeks before a -final
> kernel is released.  And once they are in Linus's tree it's usually only
> a single week before they end up in all stable kernel releases.
>
> But that's upstream, no device ships with upstream, they ship a distro
> kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
> device id into their kernels takes what, a day or two?  And that is what
> really matters as that is what goes out the door for their device.
>
> At least that is the process for when _I_ used to work on pre-installed
> Linux on devices, maybe things have gotten a lot worse since I left that
> business, but I would sure hope it wouldn't get magnitudes worse.
>
> So 2-3 months seems really long to me.
>

I should add that one thing that's really, really nice about Linux is
that, when you install it on a supported laptop, it works.  On
Windows, you end up with a horrible pile of bloated, buggy,
unsupported userspace crapware to make random buttons work.

While I'm all for improving the managability situation on Linux, let's
please make sure we don't regress the ordinary laptop functionality
story and make it as bad as it is on Windows.  We're currently in a
surprising situation in which laptops frequently work *better* on
Linux than Windows, and I think we should preserve that.

RE: WMI and Kernel:User interface

[Mario.Limonciello]

> -----Original Message-----
> From: Andy Lutomirski [mailto:luto@kernel.org]
> Sent: Tuesday, June 13, 2017 10:57 AM
> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Darren Hart <dvhart@infradead.org>; Christoph Hellwig <hch@infradead.org>;
> Pali Rohár <pali.rohar@gmail.com>; Linus Torvalds <torvalds@linux-
> foundation.org>; Limonciello, Mario <Mario_Limonciello@Dell.com>; Andy
> Shevchenko <andriy.shevchenko@linux.intel.com>; Rafael Wysocki
> <rjw@rjwysocki.net>; LKML <linux-kernel@vger.kernel.org>; platform-driver-
> x86@vger.kernel.org
> Subject: Re: WMI and Kernel:User interface
> 
> On Tue, Jun 13, 2017 at 8:50 AM, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
> > On Tue, Jun 13, 2017 at 08:38:57AM -0700, Darren Hart wrote:
> >> On Tue, Jun 13, 2017 at 12:05:35AM -0700, Christoph Hellwig wrote:
> >> > Hi Darren,
> >> >
> >> > first - can you please properly trim your replies and don't write
> >> > more than 7 characters per line?
> >>
> >> Sure... (although I think you've done all the necessary pruning for this
> >> response). 70 I presume you mean? I usually have tw set to 72...
> >> apparently I dropped that setting at some point. Will correct.
> >>
> >> >
> >> > On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> >> > > This is a big topic for sure. Speed and scale of platform enabling is something
> >> > > I would like to see us support better. The barrier to entry to kernel
> >> > > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> >> > > which would ideally, IMHO, be in the hands of the OEMs.
> >> >
> >> > It's not.  It's a trivial patch, and you cover all Linux users.  Very
> >> > much unlike say the windows world where you are stuck with installing
> >> > a vendor specific set of drivers forever.
> >> >
> >>
> >> The patch is trivial, but the process is time consuming. Two to Three
> >> months to see an ID added and released is big blocker for contemporary
> >> life cycles.
> >
> > Wait, what?  Please explain.
> >
> > Yes, it could take worse case 2-3 months to add a new device id, but
> > does it really?  I take new device ids up until 2 weeks before a -final
> > kernel is released.  And once they are in Linus's tree it's usually only
> > a single week before they end up in all stable kernel releases.
> >
> > But that's upstream, no device ships with upstream, they ship a distro
> > kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
> > device id into their kernels takes what, a day or two?  And that is what
> > really matters as that is what goes out the door for their device.
> >
> > At least that is the process for when _I_ used to work on pre-installed
> > Linux on devices, maybe things have gotten a lot worse since I left that
> > business, but I would sure hope it wouldn't get magnitudes worse.
> >
> > So 2-3 months seems really long to me.
> >
> 

Unless you catch the cycle just right, 8 weeks is normal for the delay to 
get a patch through a distro to an end user's machine.
They go through their own stable testing and let it bake before it's promoted.

> I should add that one thing that's really, really nice about Linux is
> that, when you install it on a supported laptop, it works.  On
> Windows, you end up with a horrible pile of bloated, buggy,
> unsupported userspace crapware to make random buttons work.
> 
> While I'm all for improving the managability situation on Linux, let's
> please make sure we don't regress the ordinary laptop functionality
> story and make it as bad as it is on Windows.  We're currently in a
> surprising situation in which laptops frequently work *better* on
> Linux than Windows, and I think we should preserve that.

So it's worth mentioning the main impetus for this (at least from my
company's perspective - but I expect others would share it) is to
make manageability work out of the box without having to install
additional proprietary tools.

Getting there requires work in a few areas:
1) The OEM/IBV exports the methods and the MOF (in binary form)
that describes the objects that can be interacted with and what
kind of data needs to be sent.
2) Kernel needs to provide a way to userspace to get to this data
3) A userspace OMI provider needs to be developed to read the 
MOF and load the accessible objects into a repository.
4) OMI tools can then interact with the OMI repository.

On the Windows side, you can "in-box" do manageability if the
OEM has done <1> this way.  You can use tools like powershell
to directly interact with the OMI repository.

Why don't we want the same thing on the Linux side?  If you put
up extra barriers to have to add ID's to a whitelist you are introducing
delay and artificially making it more difficult for what?

Re: WMI and Kernel:User interface

[Greg KH]

On Tue, Jun 13, 2017 at 04:12:41PM +0000, Mario.Limonciello@dell.com wrote:
> > > So 2-3 months seems really long to me.
> > >
> > 
> 
> Unless you catch the cycle just right, 8 weeks is normal for the delay to 
> get a patch through a distro to an end user's machine.
> They go through their own stable testing and let it bake before it's promoted.

Then your distro sucks :)

And for a preinstall, you know this info way in advance of getting the
device out the door with the installed distro on it, right?  If not, you
need to work with a better company...

> > I should add that one thing that's really, really nice about Linux is
> > that, when you install it on a supported laptop, it works.  On
> > Windows, you end up with a horrible pile of bloated, buggy,
> > unsupported userspace crapware to make random buttons work.
> > 
> > While I'm all for improving the managability situation on Linux, let's
> > please make sure we don't regress the ordinary laptop functionality
> > story and make it as bad as it is on Windows.  We're currently in a
> > surprising situation in which laptops frequently work *better* on
> > Linux than Windows, and I think we should preserve that.
> 
> So it's worth mentioning the main impetus for this (at least from my
> company's perspective - but I expect others would share it) is to
> make manageability work out of the box without having to install
> additional proprietary tools.

Yeah, I think we all agree on this.

> Getting there requires work in a few areas:
> 1) The OEM/IBV exports the methods and the MOF (in binary form)
> that describes the objects that can be interacted with and what
> kind of data needs to be sent.

What do you mean by this?

> 2) Kernel needs to provide a way to userspace to get to this data

Why, what can userspace do with this?

> 3) A userspace OMI provider needs to be developed to read the 
> MOF and load the accessible objects into a repository.
> 4) OMI tools can then interact with the OMI repository.

English explaination for all of this please?

What does this have to do with hooking up the wifi and brightness keys?
What about the led controls?  What else is exposed here?

> On the Windows side, you can "in-box" do manageability if the
> OEM has done <1> this way.  You can use tools like powershell
> to directly interact with the OMI repository.

You ship powershell scripts to users?  And what do those scripts do?

> Why don't we want the same thing on the Linux side?  If you put
> up extra barriers to have to add ID's to a whitelist you are introducing
> delay and artificially making it more difficult for what?

I'm not trying to create any barriers, go add a blanket "all DELL
devices" to the whitelist now if you know what is going to be in your
future product path :)

Otherwise you will always have to add something to the kernel, as you
are creating brand new hardware interfaces all the time, and the
kernel's job is to mediate them and expose them to userspace in a common
manner.

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Pali Rohár]

[-- Attachment #1: Type: Text/Plain, Size: 1919 bytes --]

Hi Greg! I will try to explain that problem with calling WMI functions.

On Tuesday 13 June 2017 18:57:14 Greg KH wrote:
> > Getting there requires work in a few areas:
> > 1) The OEM/IBV exports the methods and the MOF (in binary form)
> > that describes the objects that can be interacted with and what
> > kind of data needs to be sent.
> 
> What do you mean by this?

In ACPI is stored buffer with binary MOF data. Those data contains 
description WMI object system where are classes with methods and 
attributes (similar to C++). Then there are stored mapping from 
particular class method/attribute to WMI id.

If you want to call WMI method (which is basically ACPI method), you 
need to know WMI id and call specific ACPI method with that id and other 
magic.

Taking those magic values is not easy and WMI kernel drivers just 
reverse engineered them from ACPI bytecode or other method.

But correct way is to parse that binary MOF data and build mapping from 
classes, methods and attributes to WMI ids (needed for raw ACPI call).

Problem is that format of those binary MOF data is unknown (generated by 
program mofcomp.exe available in all MS Windows systems), but finally I 
was able to guess its structure and write "decompiler".

Search for email with subject "Binary MOF buffer in WMI is finally 
decoded!" for more details which I sent week ago to LKML.

> > 2) Kernel needs to provide a way to userspace to get to this data
> 
> Why, what can userspace do with this?

Userspace want to call WMI method "Method1" from WMI class "Class1" from 
namespace "Namespace1". And wants to know signature of that Method1, 
what are input arguments, what are outputs...

Those MOF data contains descriptions of arguments and also information 
which ACPI method with which WMI id needs to be called to execute above 
"Method1".

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 08:56:42AM -0700, Andy Lutomirski wrote:
> On Tue, Jun 13, 2017 at 8:50 AM, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
> > On Tue, Jun 13, 2017 at 08:38:57AM -0700, Darren Hart wrote:
> >> On Tue, Jun 13, 2017 at 12:05:35AM -0700, Christoph Hellwig wrote:
> >> > Hi Darren,
> >> >
> >> > first - can you please properly trim your replies and don't write
> >> > more than 7 characters per line?
> >>
> >> Sure... (although I think you've done all the necessary pruning for this
> >> response). 70 I presume you mean? I usually have tw set to 72...
> >> apparently I dropped that setting at some point. Will correct.
> >>
> >> >
> >> > On Mon, Jun 12, 2017 at 06:24:35PM -0700, Darren Hart wrote:
> >> > > This is a big topic for sure. Speed and scale of platform enabling is something
> >> > > I would like to see us support better. The barrier to entry to kernel
> >> > > changes is high, especially for trivial things, like adding IDs, GUIDs, etc.
> >> > > which would ideally, IMHO, be in the hands of the OEMs.
> >> >
> >> > It's not.  It's a trivial patch, and you cover all Linux users.  Very
> >> > much unlike say the windows world where you are stuck with installing
> >> > a vendor specific set of drivers forever.
> >> >
> >>
> >> The patch is trivial, but the process is time consuming. Two to Three
> >> months to see an ID added and released is big blocker for contemporary
> >> life cycles.
> >
> > Wait, what?  Please explain.
> >
> > Yes, it could take worse case 2-3 months to add a new device id, but
> > does it really?  I take new device ids up until 2 weeks before a -final
> > kernel is released.  And once they are in Linus's tree it's usually only
> > a single week before they end up in all stable kernel releases.
> >
> > But that's upstream, no device ships with upstream, they ship a distro
> > kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
> > device id into their kernels takes what, a day or two?  And that is what
> > really matters as that is what goes out the door for their device.
> >
> > At least that is the process for when _I_ used to work on pre-installed
> > Linux on devices, maybe things have gotten a lot worse since I left that
> > business, but I would sure hope it wouldn't get magnitudes worse.
> >
> > So 2-3 months seems really long to me.
> >
> 
> I should add that one thing that's really, really nice about Linux is
> that, when you install it on a supported laptop, it works.  On
> Windows, you end up with a horrible pile of bloated, buggy,
> unsupported userspace crapware to make random buttons work.

This argument, also made by others, seems to assume that if it isn't in the
kernel it will be of poor quality. What I see a lot of in the platform
driver bug reports are "I am not a kernel developer, ...". I believe we
would see more participation in platform specific enabling, if more of
it could be developed in userspace. That doesn't preclude a well
architected, fully open source solution.

> While I'm all for improving the managability situation on Linux, let's
> please make sure we don't regress the ordinary laptop functionality
> story and make it as bad as it is on Windows.  We're currently in a
> surprising situation in which laptops frequently work *better* on
> Linux than Windows, and I think we should preserve that.
> 

I guess I haven't been running windows on enough laptops to make a good
comparison :-D

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 05:50:00PM +0200, Greg Kroah-Hartman wrote:
> > The patch is trivial, but the process is time consuming. Two to Three
> > months to see an ID added and released is big blocker for contemporary
> > life cycles.
> 
> Wait, what?  Please explain.
> 
> Yes, it could take worse case 2-3 months to add a new device id, but
> does it really?  I take new device ids up until 2 weeks before a -final
> kernel is released.  And once they are in Linus's tree it's usually only
> a single week before they end up in all stable kernel releases.

Up to 2-3 months, the average would be shorter of course, as you say.

> 
> But that's upstream, no device ships with upstream, they ship a distro
> kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
> device id into their kernels takes what, a day or two?  And that is what
> really matters as that is what goes out the door for their device.

I'd be very interested to hear more accounts of timeline to distro
inclusion - if it is truly 1-2 days, I need to re-evaluate my position
on this. I don't see how it could be without that being done out of band
with mainline, which would in turn accrue technical debt which had to
be managed each release.

> 
> At least that is the process for when _I_ used to work on pre-installed
> Linux on devices, maybe things have gotten a lot worse since I left that
> business, but I would sure hope it wouldn't get magnitudes worse.
> 
> So 2-3 months seems really long to me.

As a concrete example, Dell has specifically made the request that we
work on a solution that doesn't require them to come back to the kernel
community each time they add a WMI GUID to their BIOS. They would like
to see those GUIDs automatically exposed.

Mario, can you provide any specific information about what the pain
points were that led to this request?

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Tue, Jun 13, 2017 at 09:22:42AM -0700, Darren Hart wrote:
> On Tue, Jun 13, 2017 at 05:50:00PM +0200, Greg Kroah-Hartman wrote:
> > > The patch is trivial, but the process is time consuming. Two to Three
> > > months to see an ID added and released is big blocker for contemporary
> > > life cycles.
> > 
> > Wait, what?  Please explain.
> > 
> > Yes, it could take worse case 2-3 months to add a new device id, but
> > does it really?  I take new device ids up until 2 weeks before a -final
> > kernel is released.  And once they are in Linus's tree it's usually only
> > a single week before they end up in all stable kernel releases.
> 
> Up to 2-3 months, the average would be shorter of course, as you say.
> 
> > 
> > But that's upstream, no device ships with upstream, they ship a distro
> > kernel.  Look at the pre-installs from SuSE and Canonical, to get a new
> > device id into their kernels takes what, a day or two?  And that is what
> > really matters as that is what goes out the door for their device.
> 
> I'd be very interested to hear more accounts of timeline to distro
> inclusion - if it is truly 1-2 days, I need to re-evaluate my position
> on this. I don't see how it could be without that being done out of band
> with mainline, which would in turn accrue technical debt which had to
> be managed each release.

Of course you do it out-of-band for mainline.  You send it upstream and
to the distro at the same time.  The team responsible for getting the
platform up and running for the preinstall merges it into the distro
kernel that gets shipped with the device, and all is well.  The next
update that the device gets, the patch is still there and when the
kernel gets updated, it's already merged.

This is what companies do all the time, they aren't running mainline on
a preinstall :)

> > At least that is the process for when _I_ used to work on pre-installed
> > Linux on devices, maybe things have gotten a lot worse since I left that
> > business, but I would sure hope it wouldn't get magnitudes worse.
> > 
> > So 2-3 months seems really long to me.
> 
> As a concrete example, Dell has specifically made the request that we
> work on a solution that doesn't require them to come back to the kernel
> community each time they add a WMI GUID to their BIOS. They would like
> to see those GUIDs automatically exposed.

What do you mean exactly by "exposed"?  What do they do with these?  Why
isn't the Dell pre-install team sending patches for this like the
Windows preinstall team is doing for their hacked-to-hell copy of
Windows?  :)

Do you have an example patch of something that was needed to get a Dell
laptop working for a new device id that didn't work this way?

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 06:52:47PM +0200, Greg Kroah-Hartman wrote:
> > As a concrete example, Dell has specifically made the request that we
> > work on a solution that doesn't require them to come back to the kernel
> > community each time they add a WMI GUID to their BIOS. They would like
> > to see those GUIDs automatically exposed.
> 
> What do you mean exactly by "exposed"?  What do they do with these?  Why

By exposed I meant: the chardev for the WMI GUID is created

The idea being the kernel maps WMI GUIDs to chardevs and shepherds the
userspace calls through to the ACPI method evaluation and back. But the
kernel wmi driver doesn't, in general, have specific knowledge of the
methods or input and output formats.

The existing drivers being the exception to "specific knowledge", and
the cause of all this filter/proxy discussion. I think we have enough
that we can put together an initial patch series, and then discuss it
there.

> isn't the Dell pre-install team sending patches for this like the
> Windows preinstall team is doing for their hacked-to-hell copy of
> Windows?  :)
> 
> Do you have an example patch of something that was needed to get a Dell
> laptop working for a new device id that didn't work this way?

Per Mario's comment, it sounds like they are and it does work this way.
It takes 8 weeks, and they don't see a reason to go through this for WMI
GUIDs.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Greg Kroah-Hartman]

On Tue, Jun 13, 2017 at 10:07:19AM -0700, Darren Hart wrote:
> On Tue, Jun 13, 2017 at 06:52:47PM +0200, Greg Kroah-Hartman wrote:
> > > As a concrete example, Dell has specifically made the request that we
> > > work on a solution that doesn't require them to come back to the kernel
> > > community each time they add a WMI GUID to their BIOS. They would like
> > > to see those GUIDs automatically exposed.
> > 
> > What do you mean exactly by "exposed"?  What do they do with these?  Why
> 
> By exposed I meant: the chardev for the WMI GUID is created
> 
> The idea being the kernel maps WMI GUIDs to chardevs and shepherds the
> userspace calls through to the ACPI method evaluation and back. But the
> kernel wmi driver doesn't, in general, have specific knowledge of the
> methods or input and output formats.

Hah, and those people who insist on "secure boot" are going to allow
userspace access to ACPI methods like this?  Well, I guess as Windows
does it, it must be ok...

I'll shut up now and just wait for patches :)

thanks,

greg k-h

Re: WMI and Kernel:User interface

[Andy Lutomirski]

On Tue, Jun 13, 2017 at 9:38 PM, Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
> On Tue, Jun 13, 2017 at 10:07:19AM -0700, Darren Hart wrote:
>> On Tue, Jun 13, 2017 at 06:52:47PM +0200, Greg Kroah-Hartman wrote:
>> > > As a concrete example, Dell has specifically made the request that we
>> > > work on a solution that doesn't require them to come back to the kernel
>> > > community each time they add a WMI GUID to their BIOS. They would like
>> > > to see those GUIDs automatically exposed.
>> >
>> > What do you mean exactly by "exposed"?  What do they do with these?  Why
>>
>> By exposed I meant: the chardev for the WMI GUID is created
>>
>> The idea being the kernel maps WMI GUIDs to chardevs and shepherds the
>> userspace calls through to the ACPI method evaluation and back. But the
>> kernel wmi driver doesn't, in general, have specific knowledge of the
>> methods or input and output formats.
>
> Hah, and those people who insist on "secure boot" are going to allow
> userspace access to ACPI methods like this?  Well, I guess as Windows
> does it, it must be ok...
>
> I'll shut up now and just wait for patches :)
>

Darren, Pali, etc, I wanted to throw out another idea for handling some of this
stuff without risking blobification: make WMI button work declaratively.  Here's
what I mean:

Currently, we have a whole bunch of WMI drivers that have a bunch of code
to do more or less the same thing.  They bind to a WMI event device,
parse events,
turn them into scancodes, and feed them to sparse_keymap_report_event.

What if we created a single driver that did this generically?  The core would
filter a WMI event device and look up each event by trying to match
it to a pattern.  A pattern is literally a bunch of bytes, where 1-4
of those bytes
are all NULL in the pattern but are marked as containing a scancode.  If there's
a match, the result is fed to sparse_keymap_report_event().  A pattern
would look
like (totally made up):

05 74 AB [0] [1] FF [2] [3]

[0] means that this byte is the low byte of the scancode.  [1] means
second byte of the scancode.  05 is a literal.

The scancode would also be programmable.

Then we let WMI drivers bind the device and feed the core a sparse
keymap and a pattern or list of patterns.  We'd also let the keymap
and the patterns be queried and edited from userspace.  Then, if a new
laptop comes out, some hwdb could feed the pattern to the kernel, and
the kernel would log the pattern that was fed in.  Then, later on, we
could update the kernel driver, or we could just leave the whole
pattern list in userspace.

Hmm?

Re: WMI and Kernel:User interface

[Darren Hart]

On Mon, Jun 19, 2017 at 03:10:09PM -0700, Andy Lutomirski wrote:
> On Tue, Jun 13, 2017 at 9:38 PM, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
> > On Tue, Jun 13, 2017 at 10:07:19AM -0700, Darren Hart wrote:
> >> On Tue, Jun 13, 2017 at 06:52:47PM +0200, Greg Kroah-Hartman wrote:
> >> > > As a concrete example, Dell has specifically made the request that we
> >> > > work on a solution that doesn't require them to come back to the kernel
> >> > > community each time they add a WMI GUID to their BIOS. They would like
> >> > > to see those GUIDs automatically exposed.
> >> >
> >> > What do you mean exactly by "exposed"?  What do they do with these?  Why
> >>
> >> By exposed I meant: the chardev for the WMI GUID is created
> >>
> >> The idea being the kernel maps WMI GUIDs to chardevs and shepherds the
> >> userspace calls through to the ACPI method evaluation and back. But the
> >> kernel wmi driver doesn't, in general, have specific knowledge of the
> >> methods or input and output formats.
> >
> > Hah, and those people who insist on "secure boot" are going to allow
> > userspace access to ACPI methods like this?  Well, I guess as Windows
> > does it, it must be ok...
> >
> > I'll shut up now and just wait for patches :)
> >
> 
> Darren, Pali, etc, I wanted to throw out another idea for handling some of this
> stuff without risking blobification: make WMI button work declaratively.  Here's
> what I mean:
> 
> Currently, we have a whole bunch of WMI drivers that have a bunch of code
> to do more or less the same thing.  They bind to a WMI event device,
> parse events,
> turn them into scancodes, and feed them to sparse_keymap_report_event.
> 
> What if we created a single driver that did this generically?  The core would
> filter a WMI event device and look up each event by trying to match
> it to a pattern.  A pattern is literally a bunch of bytes, where 1-4
> of those bytes
> are all NULL in the pattern but are marked as containing a scancode.  If there's
> a match, the result is fed to sparse_keymap_report_event().  A pattern
> would look
> like (totally made up):
> 
> 05 74 AB [0] [1] FF [2] [3]
> 
> [0] means that this byte is the low byte of the scancode.  [1] means
> second byte of the scancode.  05 is a literal.
> 
> The scancode would also be programmable.
> 
> Then we let WMI drivers bind the device and feed the core a sparse
> keymap and a pattern or list of patterns.  We'd also let the keymap
> and the patterns be queried and edited from userspace.  Then, if a new
> laptop comes out, some hwdb could feed the pattern to the kernel, and
> the kernel would log the pattern that was fed in.  Then, later on, we
> could update the kernel driver, or we could just leave the whole
> pattern list in userspace.
> 
> Hmm?
> 

Hi Andy,

So here is what the documentation [1] says about events, in part:
-----
WMI allows notifications of significant events to be delivered to interested
user-mode applications. Each type of event is uniquely named by a GUID. Events
may also carry a data block with additional information about the event. 

...

The mapping process is similar for events. The _WDG control method provides a
mapping between the WMI event GUID that represents the event and the
notification code specified in the ASL notify instruction. For example, when
ACPI provides a callback to the mapper that a control method executed a
notify(mapper-device, 0x81) function, the mapper will look up the WMI GUID
mapped to 0x81 and use this WMI GUID in building the WMI event. Before launching
the WMI event, the mapper will evaluate _WED to retrieve any additional data
that belongs with the event.
-----

I like the idea of generalizing the work and creating this event-handler type
driver. My concern/objection is that there is nothing governing how hotkeys or
buttons are implemented with WMI. A single event could be used with a specific
scancode in the "additional data" for each hotkey/button. Alternatively, a
different event with no data could be used for each hotkey/button individually.
Other horrible stateful events could be constructed as well in which the
scancodes of event X are modified by the most recent event Y's 5th bit of
"additional data".

Let's have a look at the "additional data" structures from the various drivers.

acer_wmi:
struct event_return_value {
        u8 function;
        u8 key_num;
        u16 device_state;
        u32 reserved;
} __attribute__((packed));

asus*wmi:
ACPI_TYPE_INTEGER
Encodes keys, but also brightness values from a range.

dell-wmi:
Type and code, where type may be, for example, "sequence" and then each byte of
buffen length is processed as a key.

hp-wmi:
Event ID and Data, data size varies by version. Although a specific ID triggers
hotkeys, with a single scancode key for the hp_wmi_keymap.

msi-wmi:
ACPI_TYPE_INTEGER
scancode to key is straight forward, but includes a "dead zone" for a period to
avoid duplicate events, and check for acpi backlight handling prior to reporting
brightness keys.

All told, these 5 drivers use events for hotkey in very different ways. I expect
a high degree of variety across vendors. WMI was designed as a solution for
platform (model) specific features, and that's evident in the implementations we
know about. I think attempting anything more specific than the mapping driver in
feature drivers (as opposed to platform drivers) will be unsuccessful.

1. https://msdn.microsoft.com/en-us/library/windows/hardware/dn614028(v=vs.85).aspx

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Pali Rohár]

On Monday 19 June 2017 20:37:25 Darren Hart wrote:
> dell-wmi:
> Type and code, where type may be, for example, "sequence" and then each byte of
> buffen length is processed as a key.

It is more complicated. Buffer can contain more sequences (each has own
length) and each sequence can contain different hotkey + additional info.

So I doubt it would be possible to describe in context-free grammar
specification of even buffer... And something more complicated (as L0 or
L1) is not a good idea to parse in kernel.

-- 
Pali Rohár
pali.rohar@gmail.com

Re: WMI and Kernel:User interface

[Pali Rohár]

[-- Attachment #1: Type: Text/Plain, Size: 1665 bytes --]

On Tuesday 13 June 2017 17:38:57 Darren Hart wrote:
> I'll mention this again I suspect in this thread, but rather than a
> "WMI filter" we can implement a "WMI proxy". If a kernel driver
> needs to own certain WMI calls for LED or Radio management, for
> example, all such calls can be proxied through that driver. It can
> do the necessary work to update its own state, and still perform the
> requested funtion, transparent to the userspace caller. This should
> accommodate the addition of new drivers and features to kernel
> drivers, without precluding the development of userspace management
> or platform daemons.

Such WMI proxy implemented in every WMI driver has one design problem:

There would be two different kernel APIs to configure some firmware 
settings. E.g. if particular WMI method implements turning on/off radio 
devices, then functionality would be exported to userspace via:

1) standard kernel rfkill interface which is device/driver/firmware 
neutral (and any rfkill application can control it)

2) platform/firmware specific WMI method via newly standard /dev/wmi* 
interface -- and only vendor specific application could do that and it 
would work only for this one specific WMI GUID device

I do not like idea to have two kernel <--> userspace interfaces to 
control one thing, plus one interface would be platform dependent.

In my opinion any management application which want to control radio 
switches should use option 1) rfkill interface.

And I do not see reason for exporting same duplicate, but platform 
dependent interface from kernel to userspace.

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 07:16:11PM +0200, Pali Rohár wrote:
> On Tuesday 13 June 2017 17:38:57 Darren Hart wrote:
> > I'll mention this again I suspect in this thread, but rather than a
> > "WMI filter" we can implement a "WMI proxy". If a kernel driver
> > needs to own certain WMI calls for LED or Radio management, for
> > example, all such calls can be proxied through that driver. It can
> > do the necessary work to update its own state, and still perform the
> > requested funtion, transparent to the userspace caller. This should
> > accommodate the addition of new drivers and features to kernel
> > drivers, without precluding the development of userspace management
> > or platform daemons.
> 
> Such WMI proxy implemented in every WMI driver has one design problem:
> 
> There would be two different kernel APIs to configure some firmware 
> settings. E.g. if particular WMI method implements turning on/off radio 
> devices, then functionality would be exported to userspace via:
> 
> 1) standard kernel rfkill interface which is device/driver/firmware 
> neutral (and any rfkill application can control it)
> 
> 2) platform/firmware specific WMI method via newly standard /dev/wmi* 
> interface -- and only vendor specific application could do that and it 
> would work only for this one specific WMI GUID device

Yes, platform specific control is what WMI is for.

> I do not like idea to have two kernel <--> userspace interfaces to 
> control one thing, plus one interface would be platform dependent.
> 
> In my opinion any management application which want to control radio 
> switches should use option 1) rfkill interface.

Agreed, they should.

> And I do not see reason for exporting same duplicate, but platform 
> dependent interface from kernel to userspace.
> 

So this question boils down to: do we export WMI to userspace or not?

The WMI GUIDs and methods will not be divided across convenient Linux
subsystem boundaries allowing us to pick and choose what we export. If
we export WMI to userspace, we will be providing another means of
access. Sometimes, this may cause conflict, and the answer may just be
"don't do that".

There are plenty of other examples of things you can do to screw up the
state of your system if you have the right permissions for which the
answer is "don't do that". Consider MEM(4), SETPCI(8), ... /dev/sda ...
for example.

So we can either export them and possibly offer some means of proxying
where necessary, or we can not export them.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Pali Rohár]

[-- Attachment #1: Type: Text/Plain, Size: 3252 bytes --]

On Tuesday 13 June 2017 19:40:27 Darren Hart wrote:
> On Tue, Jun 13, 2017 at 07:16:11PM +0200, Pali Rohár wrote:
> > On Tuesday 13 June 2017 17:38:57 Darren Hart wrote:
> > > I'll mention this again I suspect in this thread, but rather than
> > > a "WMI filter" we can implement a "WMI proxy". If a kernel
> > > driver needs to own certain WMI calls for LED or Radio
> > > management, for example, all such calls can be proxied through
> > > that driver. It can do the necessary work to update its own
> > > state, and still perform the requested funtion, transparent to
> > > the userspace caller. This should accommodate the addition of
> > > new drivers and features to kernel drivers, without precluding
> > > the development of userspace management or platform daemons.
> > 
> > Such WMI proxy implemented in every WMI driver has one design
> > problem:
> > 
> > There would be two different kernel APIs to configure some firmware
> > settings. E.g. if particular WMI method implements turning on/off
> > radio devices, then functionality would be exported to userspace
> > via:
> > 
> > 1) standard kernel rfkill interface which is device/driver/firmware
> > neutral (and any rfkill application can control it)
> > 
> > 2) platform/firmware specific WMI method via newly standard
> > /dev/wmi* interface -- and only vendor specific application could
> > do that and it would work only for this one specific WMI GUID
> > device
> 
> Yes, platform specific control is what WMI is for.
> 
> > I do not like idea to have two kernel <--> userspace interfaces to
> > control one thing, plus one interface would be platform dependent.
> > 
> > In my opinion any management application which want to control
> > radio switches should use option 1) rfkill interface.
> 
> Agreed, they should.
> 
> > And I do not see reason for exporting same duplicate, but platform
> > dependent interface from kernel to userspace.
> 
> So this question boils down to: do we export WMI to userspace or not?
> 
> The WMI GUIDs and methods will not be divided across convenient Linux
> subsystem boundaries allowing us to pick and choose what we export.
> If we export WMI to userspace, we will be providing another means of
> access. Sometimes, this may cause conflict, and the answer may just
> be "don't do that".
> 
> There are plenty of other examples of things you can do to screw up
> the state of your system if you have the right permissions for which
> the answer is "don't do that". Consider MEM(4), SETPCI(8), ...
> /dev/sda ... for example.

I know. There is also iopl(3). But this nor above examples are not tools 
for such activity. (Yes, there is e.g. lspci which can be switched to 
use iopl(3), but also it is not for normal usage.)

But on the other hand proposed WMI API designed are for such usage and 
developers are directly motivated to use it.

> So we can either export them and possibly offer some means of
> proxying where necessary, or we can not export them.

I just tried to show that proposed proxy has above problem and looks 
like anti-pattern for linux kernel. As this should be evaluated when 
going to accept or reject it.

-- 
Pali Rohár
pali.rohar@gmail.com

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 08:00:30PM +0200, Pali Rohár wrote:
> 
> I just tried to show that proposed proxy has above problem and looks 
> like anti-pattern for linux kernel. As this should be evaluated when 
> going to accept or reject it.

OK, I think at this point we need to follow Greg's advice and get some
code out there for review. I don't think we're going to make more
progress in pre-code discussions.

Thanks for all the time and effort everyone has put into the discussion.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Bernd Petrovitsch]

On Tue, 2017-06-13 at 10:40 -0700, Darren Hart wrote:
[...]
> There are plenty of other examples of things you can do to screw up the
> state of your system if you have the right permissions for which the
> answer is "don't do that". Consider MEM(4), SETPCI(8), ... /dev/sda ...
> for example.

So what is the problem?!
And that pretty much answers it for the Unix world IMHO (FWIW) ...

And there is *a lot* more of that if you are root.
WTF - that actually is the job definition of root/administrator/
supervisor/... to be able to fix (almost) everything and that implies
practically the rights to be able to do everything.

MfG,
	Bernd
-- 
Bernd Petrovitsch                  Email : bernd@petrovitsch.priv.at
                     LUGA : http://www.luga.at

Re: WMI and Kernel:User interface

[Pali Rohár]

On Monday 12 June 2017 18:24:35 Darren Hart wrote:
> > One thing is design of some system, another thing is real usage. We know 
> > that WMI is used for reporting events when some keys are pressed (e.g. 
> > Fn keys) or used for enabling/disabling of radio devices or controlling 
> > keyboard/LID backlight. All those parts are in Linux world handled by 
> > kernel (not userspace) and therefore WMI needs to be handled in Linux 
> > kernel.
> 
> These are examples of the subsystems I was referring to below. We can already
> assign LED triggers for some things. However, when the WMI method is
> specifically used as the mechanism for toggling the LED... yes, you make a good
> point.

I wanted to show that there are WMI methods doing functionality which is
handled by Linux kernel. And therefore it must be implemented in kernel.

If function of that WMI method is in Windows world handled by userspace,
it does not mean that in Linux we need to create way how to handle such
thing in userspace -- argument "WMI was designed to access firmware
functions from userspace" is not relevant in such Linux scenario.

> Ideally, we would provide a generic way for users/OEMs/vendors to successfully
> support and maintain their own platforms, ideally with as little kernel changes
> as possible. If we can get closer to that than we are today with this WMI work,
> I think that is worth the effort.

If OEM vendors are uncooperative, why should be ever create some
interface to call WMI methods? That looks like "we are going to cook
something, we do not know who will eat it nor how it should taste".

I know only Mario from Dell and their need to call SMBIOS functions
exported by one WMI GUID. Why no other OEM vendor entered into this
discussion?

> > > Arguably, implementing platform support with WMI through Linux kernel
> > > modules slows platform support as it is hindered by the barrier to
> > > entry and the kernel's release process - while a generic WMI
> > 
> > But we can say mainline kernel has better code quality as some random 
> > closed source 3rd vendor application.
> 
> While I think we work pretty hard at writing quality code, the code analysis
> studies I'm aware of (such as those being developed by the SIL2Linux MP project
> for the use of the Linux kernel in safety critical systems) support this
> statement for the most core code of the Linux kernel, the stuff that
> everything needs. This doesn't hold true of all parts of the kernel, however,
> with lower use drivers showing some of the highest bug rates of all subsystems
> (along with certain filesystems).

Even if there are bugs, GPLv2 license of kernel allows anybody to fix
them.

EULA of 3rd vendor closed userspace application not only disallow it,
but also make it impossible (due to missing public sources).

> > Userspace applications, and if we are talking about WMI, would be 
> > probably 3rd party vendor closed-source binaries compiled for one or two 
> > specific Linux distributions. For me it is just "random userspace 
> > application" which I do not thing that would be preinstalled or part of 
> > Linux distributions, like it is for coreutils or X Server today.
> 
> That is certainly not the outcome we'd be aiming for.

Keep in mind that switching WMI to userspace for new features would mean
need to install that software to make "new feature" work correctly. And
user would be needed to install it from vendor after installation of
system or distributions starts to packs ton of vendor closed source
software... No I think having need for such blobs is really way to the
hell for Linux world.

> > > > Question about API:
> > > > 
> > > > Are we going to export low-level ACPI methods? Or are we going to
> > > > implement BMOF parser in kernel and export high-level
> > > > class/function API to userspace with validation of input
> > > > parameters in kernel (to prevent passing garbage from userspace to
> > > > ACPI-WMI) like it is on Windows?
> > > 
> > > What we have discussed to date is exposing each WMI device as a
> > > character device. Userspace would select the method and pass a
> > > buffer formatted per the information provided by a userspace BMOF
> > > parser. This is consistent with the design goals of WMI, per the
> > > documentation. Specifically, that the mapping driver does not have
> > > any specific knowledge about the WMI GUIDs or methods.
> > 
> > Ok.
> > 
> 
> As to the "OK fine, but what are we actually going to do..." bit...
> 
> In order to support broad enabling, we should avoid Whitelists which would
> require we add new GUIDs to the kernel as fast as vendors can add them.

Based on fact that whitelisting is trivial to manage and easy to
implement, I do not think that avoiding whitelisting it a good idea.

> If we require filtering, it should be along the lines of an ACCEPT/DENY filter,
> which only denies specific accesses. Drivers which match a WMI GUID can register
> a filter callback, returning true or false (accept or deny). This would allow
> for the creation of management tools in cooperation with the existing
> drivers, without precluding the development of userspace platform-daemons which
> could handle all aspects of the WMI interface (if they conflict with an existing
> driver, they would need to run without that driver, which would avoid loading
> the WMI filter).

Yes, agree. And if filtering for current WMI kernel driver is hard (or
impossible), then such filter implemented in WMI driver would DENY any
call from userspace.

> Possible concerns, as before, we run the risk of a new driver being written, or
> a new feature being added to an existing driver, that needs to add a filter
> which would then deny userspace accesses previously allowed. It becomes a first
> to support the platform race.

With whitelisting the risk is lower. And we can mark userspace interface
as experimental for now and make part of interface that kernel can block
arbitrary userspace request.

-- 
Pali Rohár
pali.rohar@gmail.com

Re: WMI and Kernel:User interface

[Darren Hart]

On Tue, Jun 13, 2017 at 02:51:07PM +0200, Pali Rohár wrote:
> If function of that WMI method is in Windows world handled by userspace,
> it does not mean that in Linux we need to create way how to handle such
> thing in userspace -- argument "WMI was designed to access firmware
> functions from userspace" is not relevant in such Linux scenario.

We're looping now :-) We disagree on this point. That's OK, I think I
see a solution that will address both sides coming.

> > Ideally, we would provide a generic way for users/OEMs/vendors to successfully
> > support and maintain their own platforms, ideally with as little kernel changes
> > as possible. If we can get closer to that than we are today with this WMI work,
> > I think that is worth the effort.
> 
> If OEM vendors are uncooperative, why should be ever create some
> interface to call WMI methods? That looks like "we are going to cook

This line of thinking just leads to a stalemate in my opinion.

> something, we do not know who will eat it nor how it should taste".
> 
> I know only Mario from Dell and their need to call SMBIOS functions
> exported by one WMI GUID. Why no other OEM vendor entered into this
> discussion?

I'm really happy to see the level of interaction from Dell we are
seeing.  They're doing the right thing, engaging the kernel development
community in an open environment, at the right conferences, etc. That's
enough for me to engage and work on solving a problem with them.

> 
> Even if there are bugs, GPLv2 license of kernel allows anybody to fix
> them.
> 
> EULA of 3rd vendor closed userspace application not only disallow it,
> but also make it impossible (due to missing public sources).

You're conflating issues by assuming that any user of the WMI userspace
interface will be proprietary. Sure, some will, just as some kernel
modules are binary, but that argument can be applied to any interface.

> 
> > > Userspace applications, and if we are talking about WMI, would be 
> > > probably 3rd party vendor closed-source binaries compiled for one or two 
> > > specific Linux distributions. For me it is just "random userspace 
> > > application" which I do not thing that would be preinstalled or part of 
> > > Linux distributions, like it is for coreutils or X Server today.
> > 
> > That is certainly not the outcome we'd be aiming for.
> 
> Keep in mind that switching WMI to userspace for new features would mean
> need to install that software to make "new feature" work correctly. And
> user would be needed to install it from vendor after installation of
> system or distributions starts to packs ton of vendor closed source
> software... No I think having need for such blobs is really way to the
> hell for Linux world.

Again, you're assuming that if it isn't in the kernel it will be poor
quality, binary only, and distro dependent. Even if a vendor did attempt
something like that, there is nothing preventing interested users from
creating an open source solution that would be distro agnostic. And,
with the wmi method proxy concept, we could still just do it in the
kernel.

> > 
> > As to the "OK fine, but what are we actually going to do..." bit...
> > 
> > In order to support broad enabling, we should avoid Whitelists which would
> > require we add new GUIDs to the kernel as fast as vendors can add them.
> 
> Based on fact that whitelisting is trivial to manage and easy to
> implement, I do not think that avoiding whitelisting it a good idea.

As above, it isn't just the technical complexity, which is obviously
trivial, it's the timeline. It's possible dynamic IDs may address this.
I believe the proxy method also addresses the underlying concerns.

I don't see a compelling enough reason to manage GUID whitelists.

> 
> > If we require filtering, it should be along the lines of an ACCEPT/DENY filter,
> > which only denies specific accesses. Drivers which match a WMI GUID can register
> > a filter callback, returning true or false (accept or deny). This would allow
> > for the creation of management tools in cooperation with the existing
> > drivers, without precluding the development of userspace platform-daemons which
> > could handle all aspects of the WMI interface (if they conflict with an existing
> > driver, they would need to run without that driver, which would avoid loading
> > the WMI filter).
> 
> Yes, agree. And if filtering for current WMI kernel driver is hard (or
> impossible), then such filter implemented in WMI driver would DENY any
> call from userspace.
> 
> > Possible concerns, as before, we run the risk of a new driver being written, or
> > a new feature being added to an existing driver, that needs to add a filter
> > which would then deny userspace accesses previously allowed. It becomes a first
> > to support the platform race.
> 
> With whitelisting the risk is lower. And we can mark userspace interface
> as experimental for now and make part of interface that kernel can block
> arbitrary userspace request.

And finally, to the filter vs. proxy idea - phew! :-)

It occurred to me this morning while thinking about this problem and
considering Greg KH's previous comment about it's OK to change the
interface, so long as it doesn't break anything. If instead of DENYing a
method call, we allow a driver to PROXY a method call, we make the
change transparent to userspace, and still retain the ability for kernel
drivers to claim ownership of certain WMI method signatures, and ensure
consistent internal state.

When a WMI driver binds to a GUID, it can also register a
wmi_method_proxy for that GUID. When a method call is received, the
proxy is called with the method ID, input and output buffers. The proxy
can choose to handle the call itself and populate the output buffer, or
not, and let the WMI system execute it.

-- 
Darren Hart
VMware Open Source Technology Center

Re: WMI and Kernel:User interface

[Matthew Garrett]

On Tue, May 09, 2017 at 04:16:39PM -0700, Darren Hart wrote:

> To address this, I have proposed [3] that exporting WMI be opt-in, only done at
> the request of and in collaboration with a vendor, with the kernel platform
> driver given the opportunity to filter requests. This filtering would need to be
> at the method and argument inspection level, such as checking for specific bits
> in the input buffer, and rejecting the request if they conflict with an in
> kernel usage (that's worst case, in some cases just GUID or method ID could be
> sufficient).

WMI calls generally end up triggering system management mode, and SMM is 
a mess of insecure code. People have been putting extensive effort into 
avoiding mechanisms that allow root to escalate to higher privilege 
levels - this is almost certainly the opposite of that. If the filtering 
is sufficient to guarantee that no invalid input will ever hit the 
firmware then that's not a problem, but that doesn't seem meaningfully 
less complicated than just writing a proper driver in the first place.

As things stand, I think this is functionality that would have to be 
disabled by the lockdown patchset, which means that it's functionality 
that wouldn't exist for the majority of non-server platforms (and an 
increasing number of server platforms).
-- 
Matthew Garrett | mjg59@srcf.ucam.org