wrapper device driver

wrapper device driver

[riya khanna]

Hi,

I'm writing a device driver to to provide a wrapper device around a
real device. Is it acceptable to do the following:

wrapper_dev_open(flags) {
   // do additional bookkeeping
   real_dev_filp = filp_open(real_device_node_path, flags);
}

wrapper_dev_mmap(mmap_parameters) {
   // do additional checks
   return real_dev_filp->f_op->mmap(mmap_parameters);
}

wrapper_dev_ioctl(ioctl_parameters) {
   // do additional checks
   return real_dev_filp->f_op->ioctl(ioctl_parameters);
}

Is it safe to do something like this? what would be the caveats? Given
a good use case, would the maintainers be willing to mainstream
something like this? Thanks!

-Riya

FW: wrapper device driver

[Malte Vesper]

________________________________________
From: Malte Vesper
Sent: 02 February 2015 21:43
To: riya khanna
Subject: RE: wrapper device driver

Why don't you implement your wrapper as a userspace library?
________________________________________
From: kernelnewbies-bounces@kernelnewbies.org [kernelnewbies-bounces at kernelnewbies.org] on behalf of riya khanna [riyakhanna1983 at gmail.com]
Sent: 02 February 2015 21:24
To: kernelnewbies
Subject: wrapper device driver

Hi,

I'm writing a device driver to to provide a wrapper device around a
real device. Is it acceptable to do the following:

wrapper_dev_open(flags) {
   // do additional bookkeeping
   real_dev_filp = filp_open(real_device_node_path, flags);
}

wrapper_dev_mmap(mmap_parameters) {
   // do additional checks
   return real_dev_filp->f_op->mmap(mmap_parameters);
}

wrapper_dev_ioctl(ioctl_parameters) {
   // do additional checks
   return real_dev_filp->f_op->ioctl(ioctl_parameters);
}

Is it safe to do something like this? what would be the caveats? Given
a good use case, would the maintainers be willing to mainstream
something like this? Thanks!

-Riya

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

FW: wrapper device driver

[riya khanna]

I guess a userspace library approach won't be transparent to the applications.

On Mon, Feb 2, 2015 at 3:44 PM, Malte Vesper
<malte.vesper@postgrad.manchester.ac.uk> wrote:
>
> ________________________________________
> From: Malte Vesper
> Sent: 02 February 2015 21:43
> To: riya khanna
> Subject: RE: wrapper device driver
>
> Why don't you implement your wrapper as a userspace library?
> ________________________________________
> From: kernelnewbies-bounces at kernelnewbies.org [kernelnewbies-bounces at kernelnewbies.org] on behalf of riya khanna [riyakhanna1983 at gmail.com]
> Sent: 02 February 2015 21:24
> To: kernelnewbies
> Subject: wrapper device driver
>
> Hi,
>
> I'm writing a device driver to to provide a wrapper device around a
> real device. Is it acceptable to do the following:
>
> wrapper_dev_open(flags) {
>    // do additional bookkeeping
>    real_dev_filp = filp_open(real_device_node_path, flags);
> }
>
> wrapper_dev_mmap(mmap_parameters) {
>    // do additional checks
>    return real_dev_filp->f_op->mmap(mmap_parameters);
> }
>
> wrapper_dev_ioctl(ioctl_parameters) {
>    // do additional checks
>    return real_dev_filp->f_op->ioctl(ioctl_parameters);
> }
>
> Is it safe to do something like this? what would be the caveats? Given
> a good use case, would the maintainers be willing to mainstream
> something like this? Thanks!
>
> -Riya
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

FW: wrapper device driver

[Greg KH]

On Mon, Feb 02, 2015 at 04:05:35PM -0600, riya khanna wrote:
> I guess a userspace library approach won't be transparent to the applications.

Look at cuse, I think it provides what you are wanting to do here.

But as you really didn't say what your goals are, it's hard to
determine.

good luck,

greg k-h

FW: wrapper device driver

[riya khanna]

The goal is to provide multiple instances of a real device, where each
instance could be assigned to a container. This is to enable support
for device multiplexing in user space.

I did look at CUSE. However,  I realized that not all the device
driver's all all operations to be forwarded to CUSE proxy daemon -
some device drivers do bookkeeping based on process PID, so CUSE proxy
daemon cannot operate on behalf of processes. Performance is another
reason.

So would it be acceptable to modify CUSE kernel driver to
filp_open(real_device_node), and use corresponding
real_filp->f_op->operation() for the file_operations that cannot be
forwarded to or are unimplemented by userspace CUSE proxy. Target/real
device file operations would act as fallback operations for
unimplemented operations.

On Mon, Feb 2, 2015 at 4:17 PM, Greg KH <greg@kroah.com> wrote:
> On Mon, Feb 02, 2015 at 04:05:35PM -0600, riya khanna wrote:
>> I guess a userspace library approach won't be transparent to the applications.
>
> Look at cuse, I think it provides what you are wanting to do here.
>
> But as you really didn't say what your goals are, it's hard to
> determine.
>
> good luck,
>
> greg k-h

FW: wrapper device driver

[Greg KH]

A: No.
Q: Should I include quotations after my reply?

http://daringfireball.net/2007/07/on_top

On Mon, Feb 02, 2015 at 04:46:24PM -0600, riya khanna wrote:
> The goal is to provide multiple instances of a real device, where each
> instance could be assigned to a container. This is to enable support
> for device multiplexing in user space.

Heh, no, don't do it.

Seriously, don't, it's been shot down time and time again in person and
in emails.  The 2013 Plumbers conference had a whole session on this in
which people yelled at me for 45+ minutes, it was fun, I still said no.

> I did look at CUSE. However,  I realized that not all the device
> driver's all all operations to be forwarded to CUSE proxy daemon -
> some device drivers do bookkeeping based on process PID, so CUSE proxy
> daemon cannot operate on behalf of processes. Performance is another
> reason.

Have you benchmarked CUSE?  It's fast, but the real question is what
types of devices are you trying to use this for?

If a device is to be multiplexed, it needs to be done so in the driver
for the device, or the subsystem, you can't do it in a "wrapper" driver,
or even in userspace, as state will get confused and messed up and it
will not work properly in the end, sorry.

> So would it be acceptable to modify CUSE kernel driver to
> filp_open(real_device_node), and use corresponding
> real_filp->f_op->operation() for the file_operations that cannot be
> forwarded to or are unimplemented by userspace CUSE proxy. Target/real
> device file operations would act as fallback operations for
> unimplemented operations.

Nope, do it in each individual subsystem that you want to change.  But
again, see my above comments, this isn't something that is going to get
in easily, if at all, sorry.

good luck,

greg k-h

FW: wrapper device driver

[riya khanna]

On Mon, Feb 2, 2015 at 5:00 PM, Greg KH <greg@kroah.com> wrote:
>
> On Mon, Feb 02, 2015 at 04:46:24PM -0600, riya khanna wrote:
>> The goal is to provide multiple instances of a real device, where each
>> instance could be assigned to a container. This is to enable support
>> for device multiplexing in user space.
>
> Heh, no, don't do it.
>
> Seriously, don't, it's been shot down time and time again in person and
> in emails.  The 2013 Plumbers conference had a whole session on this in
> which people yelled at me for 45+ minutes, it was fun, I still said no.
>

Yes, I'm apprised of the LPC '13 and email discussion on device
namespaces. In fact the reason I started down this path is because,
like you said, the discussion outcome ruled out kernel changes.

>> I did look at CUSE. However,  I realized that not all the device
>> driver's all all operations to be forwarded to CUSE proxy daemon -
>> some device drivers do bookkeeping based on process PID, so CUSE proxy
>> daemon cannot operate on behalf of processes. Performance is another
>> reason.
>
> Have you benchmarked CUSE?  It's fast, but the real question is what
> types of devices are you trying to use this for?
>
> If a device is to be multiplexed, it needs to be done so in the driver
> for the device, or the subsystem, you can't do it in a "wrapper" driver,
> or even in userspace, as state will get confused and messed up and it
> will not work properly in the end, sorry.
>

The purpose of multiplexing is to either block undesired
events/operations on devices (e.g. input, graphics) or respond to the
applications based on the in-memory state of device instances.
With CUSE, in-memory states can be maintained and mediated in user
space. AFAIU, doing device multiplexing in the kernel would also
entail the same - maintain in-memory state (replicating data structs)
for each virtual device instance, but that also means changing the
drivers/subsystem to incorporate this functionality. I may be missing
something here, but I'm not sure why maintaining the state in
userspace (as a CUSE device) would be messy and not work. I would
appreciate more explanation. thanks.

-Riya

>> So would it be acceptable to modify CUSE kernel driver to
>> filp_open(real_device_node), and use corresponding
>> real_filp->f_op->operation() for the file_operations that cannot be
>> forwarded to or are unimplemented by userspace CUSE proxy. Target/real
>> device file operations would act as fallback operations for
>> unimplemented operations.
>
> Nope, do it in each individual subsystem that you want to change.  But
> again, see my above comments, this isn't something that is going to get
> in easily, if at all, sorry.
>
> good luck,
>
> greg k-h

FW: wrapper device driver

[Valdis.Kletnieks at vt.edu]

On Mon, 02 Feb 2015 17:50:43 -0600, riya khanna said:
> The purpose of multiplexing is to either block undesired
> events/operations on devices (e.g. input, graphics) or respond to the
> applications based on the in-memory state of device instances.

Those who don't remember the history of filtering ioctl's for CD/RW devices
are doomed to repeat it... :)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150202/3b88914b/attachment.bin 

FW: wrapper device driver

[riya khanna]

On Mon, Feb 2, 2015 at 8:49 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Mon, 02 Feb 2015 17:50:43 -0600, riya khanna said:
>> The purpose of multiplexing is to either block undesired
>> events/operations on devices (e.g. input, graphics) or respond to the
>> applications based on the in-memory state of device instances.
>
> Those who don't remember the history of filtering ioctl's for CD/RW devices
> are doomed to repeat it... :)
>

For those who don't remember the history of filtering ioctl's for
CD/RW devices, would you care to explain/point out the problem.

FW: wrapper device driver

[Greg KH]

On Mon, Feb 02, 2015 at 05:50:43PM -0600, riya khanna wrote:
> On Mon, Feb 2, 2015 at 5:00 PM, Greg KH <greg@kroah.com> wrote:
> >
> > On Mon, Feb 02, 2015 at 04:46:24PM -0600, riya khanna wrote:
> >> The goal is to provide multiple instances of a real device, where each
> >> instance could be assigned to a container. This is to enable support
> >> for device multiplexing in user space.
> >
> > Heh, no, don't do it.
> >
> > Seriously, don't, it's been shot down time and time again in person and
> > in emails.  The 2013 Plumbers conference had a whole session on this in
> > which people yelled at me for 45+ minutes, it was fun, I still said no.
> >
> 
> Yes, I'm apprised of the LPC '13 and email discussion on device
> namespaces. In fact the reason I started down this path is because,
> like you said, the discussion outcome ruled out kernel changes.

The discussion also stated that this was impossible without kernel
changes, you need to do this in the kernel on a subsystem-by-subsystem
basis, there is no "magic fix" to make all devices work at once.  That
was my main point of that discussion, people seem to be thinking that
they don't want to do the hard work for some reason :(

> >> I did look at CUSE. However,  I realized that not all the device
> >> driver's all all operations to be forwarded to CUSE proxy daemon -
> >> some device drivers do bookkeeping based on process PID, so CUSE proxy
> >> daemon cannot operate on behalf of processes. Performance is another
> >> reason.
> >
> > Have you benchmarked CUSE?  It's fast, but the real question is what
> > types of devices are you trying to use this for?
> >
> > If a device is to be multiplexed, it needs to be done so in the driver
> > for the device, or the subsystem, you can't do it in a "wrapper" driver,
> > or even in userspace, as state will get confused and messed up and it
> > will not work properly in the end, sorry.
> >
> 
> The purpose of multiplexing is to either block undesired
> events/operations on devices (e.g. input, graphics) or respond to the
> applications based on the in-memory state of device instances.

You didn't answer my question of "which specific devices do you care
about" :(

You can't "filter" device commands (see the before-mentioned cdrom mess,
you have learned from history, right?)  And you can't assume you know
what the in-kernel state of devices really are, as they are getting
commands from the hardware itself that changes this state, not all
changes come from userspace.

> With CUSE, in-memory states can be maintained and mediated in user
> space. AFAIU, doing device multiplexing in the kernel would also
> entail the same - maintain in-memory state (replicating data structs)
> for each virtual device instance, but that also means changing the
> drivers/subsystem to incorporate this functionality. I may be missing
> something here, but I'm not sure why maintaining the state in
> userspace (as a CUSE device) would be messy and not work. I would
> appreciate more explanation. thanks.

It's impossible to maintain the state in userspace properly.  If you
could do it, then you would just have a userspace device driver, and why
need the kernel at all for it?  :)

Think through the specifics of a specific device you wish to try to
mitigate access to, and walk through the complexity and marvel at how
much more work you are now doing than the original kernel driver did.

Again, this has to be done on a subsystem-basis, in the kernel, for it
to work properly.  And when you try to do that you will get a lot of
pushback, which is correct, as you will be adding complexity for almost
no gain in the end.

Just properly assign different devices to different containers, if you
want to do more than that, then think about using a "real" virtual
machine, which does properly abstract the hardware away like this (or
really, it just does hardware pass-through and again, does not share the
hardware fully, look at how USB works in virtual machines for examples.)

good luck,

greg k-h

FW: wrapper device driver

[riya khanna]

On Mon, Feb 2, 2015 at 9:15 PM, Greg KH <greg@kroah.com> wrote:
> On Mon, Feb 02, 2015 at 05:50:43PM -0600, riya khanna wrote:
>> On Mon, Feb 2, 2015 at 5:00 PM, Greg KH <greg@kroah.com> wrote:
>> >
>> > On Mon, Feb 02, 2015 at 04:46:24PM -0600, riya khanna wrote:
>> >> The goal is to provide multiple instances of a real device, where each
>> >> instance could be assigned to a container. This is to enable support
>> >> for device multiplexing in user space.
>> >
>> > Heh, no, don't do it.
>> >
>> > Seriously, don't, it's been shot down time and time again in person and
>> > in emails.  The 2013 Plumbers conference had a whole session on this in
>> > which people yelled at me for 45+ minutes, it was fun, I still said no.
>> >
>>
>> Yes, I'm apprised of the LPC '13 and email discussion on device
>> namespaces. In fact the reason I started down this path is because,
>> like you said, the discussion outcome ruled out kernel changes.
>
> The discussion also stated that this was impossible without kernel
> changes, you need to do this in the kernel on a subsystem-by-subsystem
> basis, there is no "magic fix" to make all devices work at once.  That
> was my main point of that discussion, people seem to be thinking that
> they don't want to do the hard work for some reason :(
>
>> >> I did look at CUSE. However,  I realized that not all the device
>> >> driver's all all operations to be forwarded to CUSE proxy daemon -
>> >> some device drivers do bookkeeping based on process PID, so CUSE proxy
>> >> daemon cannot operate on behalf of processes. Performance is another
>> >> reason.
>> >
>> > Have you benchmarked CUSE?  It's fast, but the real question is what
>> > types of devices are you trying to use this for?
>> >
>> > If a device is to be multiplexed, it needs to be done so in the driver
>> > for the device, or the subsystem, you can't do it in a "wrapper" driver,
>> > or even in userspace, as state will get confused and messed up and it
>> > will not work properly in the end, sorry.
>> >
>>
>> The purpose of multiplexing is to either block undesired
>> events/operations on devices (e.g. input, graphics) or respond to the
>> applications based on the in-memory state of device instances.
>
> You didn't answer my question of "which specific devices do you care
> about" :(
>
> You can't "filter" device commands (see the before-mentioned cdrom mess,
> you have learned from history, right?)  And you can't assume you know
> what the in-kernel state of devices really are, as they are getting
> commands from the hardware itself that changes this state, not all
> changes come from userspace.
>

Thanks for the explanation and sorry for not answering your question.
I care about input and graphics subsystem (no specific device) because
there are abstractions already available in the kernel for them (e.g.
evdev, drm kms). I'm not talking about filtering device-specific
commands through ioctl(). I understand the problem with that. However,
I was wondering if one could take advantage of generic interfaces
(e.g. evdev) and mediate accesses in user space through ioctl. For
example, if a container is in the background (i.e. user not
interacting with it), then all inputs should be blocked to it, but if
it becomes active again inputs must be redirected to its virtual
devices. This can be done through evdev interface's EVIOCGRAB and
EVIOCREVOKE ioctls.

>> With CUSE, in-memory states can be maintained and mediated in user
>> space. AFAIU, doing device multiplexing in the kernel would also
>> entail the same - maintain in-memory state (replicating data structs)
>> for each virtual device instance, but that also means changing the
>> drivers/subsystem to incorporate this functionality. I may be missing
>> something here, but I'm not sure why maintaining the state in
>> userspace (as a CUSE device) would be messy and not work. I would
>> appreciate more explanation. thanks.
>
> It's impossible to maintain the state in userspace properly.  If you
> could do it, then you would just have a userspace device driver, and why
> need the kernel at all for it?  :)
>
> Think through the specifics of a specific device you wish to try to
> mitigate access to, and walk through the complexity and marvel at how
> much more work you are now doing than the original kernel driver did.
>
> Again, this has to be done on a subsystem-basis, in the kernel, for it
> to work properly.  And when you try to do that you will get a lot of
> pushback, which is correct, as you will be adding complexity for almost
> no gain in the end.
>
> Just properly assign different devices to different containers, if you
> want to do more than that, then think about using a "real" virtual
> machine, which does properly abstract the hardware away like this (or
> really, it just does hardware pass-through and again, does not share the
> hardware fully, look at how USB works in virtual machines for examples.)
>
> good luck,
>
> greg k-h

FW: wrapper device driver

[Valdis.Kletnieks at vt.edu]

On Tue, 03 Feb 2015 09:34:31 -0600, riya khanna said:

> (e.g. evdev) and mediate accesses in user space through ioctl. For
> example, if a container is in the background (i.e. user not
> interacting with it), then all inputs should be blocked to it, but if
> it becomes active again inputs must be redirected to its virtual
> devices.

That's actually nowhere near as obviously true as you might think, and depends
*heavily* on what the definition and exact semantics of "background" is. Do
you mean 'process &' type of background, or "user ctl-Z'ed the process" background,
or "screensaver has kicked in" background?  Those all have very different
semantics....



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150203/befacc04/attachment.bin 

FW: wrapper device driver

[Greg KH]

On Tue, Feb 03, 2015 at 09:34:31AM -0600, riya khanna wrote:
> > You didn't answer my question of "which specific devices do you care
> > about" :(
> >
> > You can't "filter" device commands (see the before-mentioned cdrom mess,
> > you have learned from history, right?)  And you can't assume you know
> > what the in-kernel state of devices really are, as they are getting
> > commands from the hardware itself that changes this state, not all
> > changes come from userspace.
> >
> 
> Thanks for the explanation and sorry for not answering your question.
> I care about input and graphics subsystem (no specific device) because
> there are abstractions already available in the kernel for them (e.g.
> evdev, drm kms). I'm not talking about filtering device-specific
> commands through ioctl(). I understand the problem with that. However,
> I was wondering if one could take advantage of generic interfaces
> (e.g. evdev) and mediate accesses in user space through ioctl.

You have looked at the bazillion different DRM ioctls, right?  And
understand the state machine complexities that happen between the kernel
and userspace for a DRM device?  Have you written up how you would
handle mediating that interface in a reasonable manner?

> For
> example, if a container is in the background (i.e. user not
> interacting with it), then all inputs should be blocked to it, but if
> it becomes active again inputs must be redirected to its virtual
> devices.

That seems foolish, what happens if that background container tried to
set the state of a specific device?  How can you reliably fail that?

> This can be done through evdev interface's EVIOCGRAB and
> EVIOCREVOKE ioctls.

evdev already gives you all the needed hooks to try to do something like
this from userspace.  There shouldn't need to be anything extra to do
here, otherwise, how would input devices work today in multi-session
systems?

I would go back and talk to who ever is thinking that they want this
type of solution, to just sit down and look at the DRM ioctl interface.
After they are done screaming, you shouldn't have to worry about being
given impossible tasks anymore :)

Best of luck,

greg k-h

wrapper device driver

[Valdis.Kletnieks at vt.edu]

On Mon, 02 Feb 2015 15:24:37 -0600, riya khanna said:
> Hi,
>
> I'm writing a device driver to to provide a wrapper device around a
> real device. Is it acceptable to do the following:
>
> wrapper_dev_open(flags) {
>    // do additional bookkeeping
>    real_dev_filp = filp_open(real_device_node_path, flags);
> }
>
> wrapper_dev_mmap(mmap_parameters) {
>    // do additional checks
>    return real_dev_filp->f_op->mmap(mmap_parameters);
> }
>
> wrapper_dev_ioctl(ioctl_parameters) {
>    // do additional checks
>    return real_dev_filp->f_op->ioctl(ioctl_parameters);
> }
>
> Is it safe to do something like this?

Probably not.  If you return a struct real_dev->f_op then any further
calls will vector through that structure, totally unseen to your wrapper,
which means you won't be able to provide whatever added-value extras the
wrapper is trying to do (in other words, your wrapper ends up not doing
anything).

The real fun starts when something calls real->f_op->close() out from
under you... :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150202/204f6285/attachment.bin 

wrapper device driver

[riya khanna]

struct real_dev->f_op would not be made available to the userspace.
It's for target/real device bookkeeping.

On Mon, Feb 2, 2015 at 4:02 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Mon, 02 Feb 2015 15:24:37 -0600, riya khanna said:
>> Hi,
>>
>> I'm writing a device driver to to provide a wrapper device around a
>> real device. Is it acceptable to do the following:
>>
>> wrapper_dev_open(flags) {
>>    // do additional bookkeeping
>>    real_dev_filp = filp_open(real_device_node_path, flags);
>> }
>>
>> wrapper_dev_mmap(mmap_parameters) {
>>    // do additional checks
>>    return real_dev_filp->f_op->mmap(mmap_parameters);
>> }
>>
>> wrapper_dev_ioctl(ioctl_parameters) {
>>    // do additional checks
>>    return real_dev_filp->f_op->ioctl(ioctl_parameters);
>> }
>>
>> Is it safe to do something like this?
>
> Probably not.  If you return a struct real_dev->f_op then any further
> calls will vector through that structure, totally unseen to your wrapper,
> which means you won't be able to provide whatever added-value extras the
> wrapper is trying to do (in other words, your wrapper ends up not doing
> anything).
>
> The real fun starts when something calls real->f_op->close() out from
> under you... :)