Kernel-space flirting with user-space

Kernel-space flirting with user-space

[Rajat Sharma]

If you do not have much dependency to be in kernel mode, you can implement,
or atleast initially, a user mode filesystem using fuse.
------------------------------
From: Simon
Sent: 11-01-2013 02:45
To: kernelnewbies at kernelnewbies.org
Subject: Kernel-space flirting with user-space

Hi there,

  I'm rather new to kernel development, but I am somewhat experienced with
Linux, kernel compilation, etc.  I'm just passed writing a helloworld
module, dummy filesystem (ramfs wrapper) and proc entry.  I would like to
know how communication between kernel & user-space is done (the standard)
for a high number of tiny and large messages going in both directions
(large messages would need to be transfered incrementally, in chunks, can
be sequential or random access).  I plan on developping a network
filesystem.  I would appreciate if someone could clarify where I'm wrong
and confirm where I'm right, and ideally suggest some vocabulary, source
files to lookup, online docs or even books.  The minimum you can give will
be most appreciated!  =)

  Firstly, I was wondering if it would be possible to implement the
filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
think this is really not the recommended approach, but an advantage I see
is it could be used to mount the root filesystem before calling init (that
would be before user-space exists, right?).  On the other hand, separating
the work to have a program/daemon in user-space do the communication and
processing would allow me to write that part in C++ (which I personally
prefer).

  Secondly, I wonder how I can "bind" a user-space program/daemon with the
kernel-space of the module.

  Using procfs:  It seems technically possible to achieve my goal with it,
but I feel this would not be the standard approach.  I would rather keep it
for live config & live status reporting.
  Using sysfs:  I've read too much misleading information that I'm not sure
anymore what it's for or even if it's still used or deprecated!  Can
someone clarify?  It looks technically identical to procfs, but it's
mission is a different one, correct?
  Using pipes:  This may be a good way, but again, I'm not sure if it's the
standard way.
  Using IPC:  As far as I understand, IPC would be the way if we'd be
talking of two threads within the same process, or at least within
kernel-space.  Though I am likely to be wrong.
  Using a char device:  It seems technically possible as well, but it may
be difficult to deal with if I ever have more than one user-space process.
  Using a block device:  My file system works with files and their
metadata, but not with blocks, so this is not suitable.  Though it might be
a nice experiment to build a network block device containing an ext2
filesystem (which I think would be similar to iSCSI).
  Using net interface: Not applicable.  And an experiment to try to make it
applicable doesn't even seem remotely fun.  ;)

  I've seen the functions copy_to/from_user() or something like that, but I
haven't seen the user-space counter-part of these.  I think this may
actually be the best approach by far, but I really lack info/docs to go
that way.  The truly best way might be to use shared memory to avoid
copying data internally, but I haven't looked into that much yet.

  Finally, is it possible/correct/standard to do something similar to the
following for the local kernel & user-space communication?  (I haven't
reviewed the syntax, so take it as some kind of pseudo-code!)

struct foo {...} outgoingFoo;
// foo contains no pointers, so it is of a fixed size always.
fwrite(&outgoingFoo, 1, sizeof(outgoingFoo), output);
  [...then on the other end...]
if(sizeof(fread( buffer, 1, buffer_size, input);
struct foo * incomingFoo = (struct foo *) buffer;

Regards,
  Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130110/44cd14aa/attachment-0001.html 

Kernel-space flirting with user-space

[Simon]

Hi there,

  I'm rather new to kernel development, but I am somewhat experienced with
Linux, kernel compilation, etc.  I'm just passed writing a helloworld
module, dummy filesystem (ramfs wrapper) and proc entry.  I would like to
know how communication between kernel & user-space is done (the standard)
for a high number of tiny and large messages going in both directions
(large messages would need to be transfered incrementally, in chunks, can
be sequential or random access).  I plan on developping a network
filesystem.  I would appreciate if someone could clarify where I'm wrong
and confirm where I'm right, and ideally suggest some vocabulary, source
files to lookup, online docs or even books.  The minimum you can give will
be most appreciated!  =)

  Firstly, I was wondering if it would be possible to implement the
filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
think this is really not the recommended approach, but an advantage I see
is it could be used to mount the root filesystem before calling init (that
would be before user-space exists, right?).  On the other hand, separating
the work to have a program/daemon in user-space do the communication and
processing would allow me to write that part in C++ (which I personally
prefer).

  Secondly, I wonder how I can "bind" a user-space program/daemon with the
kernel-space of the module.

  Using procfs:  It seems technically possible to achieve my goal with it,
but I feel this would not be the standard approach.  I would rather keep it
for live config & live status reporting.
  Using sysfs:  I've read too much misleading information that I'm not sure
anymore what it's for or even if it's still used or deprecated!  Can
someone clarify?  It looks technically identical to procfs, but it's
mission is a different one, correct?
  Using pipes:  This may be a good way, but again, I'm not sure if it's the
standard way.
  Using IPC:  As far as I understand, IPC would be the way if we'd be
talking of two threads within the same process, or at least within
kernel-space.  Though I am likely to be wrong.
  Using a char device:  It seems technically possible as well, but it may
be difficult to deal with if I ever have more than one user-space process.
  Using a block device:  My file system works with files and their
metadata, but not with blocks, so this is not suitable.  Though it might be
a nice experiment to build a network block device containing an ext2
filesystem (which I think would be similar to iSCSI).
  Using net interface: Not applicable.  And an experiment to try to make it
applicable doesn't even seem remotely fun.  ;)

  I've seen the functions copy_to/from_user() or something like that, but I
haven't seen the user-space counter-part of these.  I think this may
actually be the best approach by far, but I really lack info/docs to go
that way.  The truly best way might be to use shared memory to avoid
copying data internally, but I haven't looked into that much yet.

  Finally, is it possible/correct/standard to do something similar to the
following for the local kernel & user-space communication?  (I haven't
reviewed the syntax, so take it as some kind of pseudo-code!)

struct foo {...} outgoingFoo;
// foo contains no pointers, so it is of a fixed size always.
fwrite(&outgoingFoo, 1, sizeof(outgoingFoo), output);
  [...then on the other end...]
if(sizeof(fread( buffer, 1, buffer_size, input);
struct foo * incomingFoo = (struct foo *) buffer;

Regards,
  Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130110/101b7b71/attachment.html 

Kernel-space flirting with user-space

[devendra.aaru]

On Thu, Jan 10, 2013 at 4:09 PM, Simon <turner25@gmail.com> wrote:
> Hi there,
>
>   I'm rather new to kernel development, but I am somewhat experienced with
> Linux, kernel compilation, etc.  I'm just passed writing a helloworld
> module, dummy filesystem (ramfs wrapper) and proc entry.  I would like to
> know how communication between kernel & user-space is done (the standard)
> for a high number of tiny and large messages going in both directions (large
> messages would need to be transfered incrementally, in chunks, can be
> sequential or random access).  I plan on developping a network filesystem.
> I would appreciate if someone could clarify where I'm wrong and confirm
> where I'm right, and ideally suggest some vocabulary, source files to
> lookup, online docs or even books.  The minimum you can give will be most
> appreciated!  =)
>
>   Firstly, I was wondering if it would be possible to implement the
> filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
> think this is really not the recommended approach, but an advantage I see is
> it could be used to mount the root filesystem before calling init (that
> would be before user-space exists, right?).  On the other hand, separating
> the work to have a program/daemon in user-space do the communication and
> processing would allow me to write that part in C++ (which I personally
> prefer).
>
>   Secondly, I wonder how I can "bind" a user-space program/daemon with the
> kernel-space of the module.
>
>   Using procfs:  It seems technically possible to achieve my goal with it,
> but I feel this would not be the standard approach.  I would rather keep it
> for live config & live status reporting.
>   Using sysfs:  I've read too much misleading information that I'm not sure
> anymore what it's for or even if it's still used or deprecated!  Can someone
> clarify?  It looks technically identical to procfs, but it's mission is a
> different one, correct?
>   Using pipes:  This may be a good way, but again, I'm not sure if it's the
> standard way.
>   Using IPC:  As far as I understand, IPC would be the way if we'd be
> talking of two threads within the same process, or at least within
> kernel-space.  Though I am likely to be wrong.
>   Using a char device:  It seems technically possible as well, but it may be
> difficult to deal with if I ever have more than one user-space process.
>   Using a block device:  My file system works with files and their metadata,
> but not with blocks, so this is not suitable.  Though it might be a nice
> experiment to build a network block device containing an ext2 filesystem
> (which I think would be similar to iSCSI).
>   Using net interface: Not applicable.  And an experiment to try to make it
> applicable doesn't even seem remotely fun.  ;)
>
>   I've seen the functions copy_to/from_user() or something like that, but I
> haven't seen the user-space counter-part of these.  I think this may
> actually be the best approach by far, but I really lack info/docs to go that
> way.  The truly best way might be to use shared memory to avoid copying data
> internally, but I haven't looked into that much yet.
>
>   Finally, is it possible/correct/standard to do something similar to the
> following for the local kernel & user-space communication?  (I haven't
> reviewed the syntax, so take it as some kind of pseudo-code!)
>
> struct foo {...} outgoingFoo;
> // foo contains no pointers, so it is of a fixed size always.
> fwrite(&outgoingFoo, 1, sizeof(outgoingFoo), output);
>   [...then on the other end...]
> if(sizeof(fread( buffer, 1, buffer_size, input);
> struct foo * incomingFoo = (struct foo *) buffer;
>
> Regards,
>   Simon
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>

What about netlink and mmap?

Kernel-space flirting with user-space

[Simon]

> >   Secondly, I wonder how I can "bind" a user-space program/daemon with
the
> > kernel-space of the module.
>
> What about netlink and mmap?

Hi Devendra,
  I glanced at netlink and I really like the idea.  I think it would allow
me to build the whole thing in kernel-space and a user-space process could
simply "proxy" the packets over TCP/IP, SSL and friends! ;)

  I also looked at mmap and while it may be used for communication, it
seems to be used for many other things.  I fear this could be confusing or
that documentation will not address my need specifically though.  An
educated guess tells me it might be more efficient than netlink however.
 I'll give it a shot later and compare.

Thanks,
  Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130116/d44c2478/attachment.html 

Kernel-space flirting with user-space

[Henrique Rodrigues]

I think the LDD book is a good starting point: http://lwn.net/Kernel/LDD3/

You can find some examples of kernel/user communication here:
http://people.ee.ethz.ch/~arkeller/linux/kernel_user_space_howto.html

-- 
Henrique Rodrigues
http://www.dcc.ufmg.br/~hsr

On Thu, Jan 10, 2013 at 1:09 PM, Simon <turner25@gmail.com> wrote:

> Hi there,
>
>   I'm rather new to kernel development, but I am somewhat experienced with
> Linux, kernel compilation, etc.  I'm just passed writing a helloworld
> module, dummy filesystem (ramfs wrapper) and proc entry.  I would like to
> know how communication between kernel & user-space is done (the standard)
> for a high number of tiny and large messages going in both directions
> (large messages would need to be transfered incrementally, in chunks, can
> be sequential or random access).  I plan on developping a network
> filesystem.  I would appreciate if someone could clarify where I'm wrong
> and confirm where I'm right, and ideally suggest some vocabulary, source
> files to lookup, online docs or even books.  The minimum you can give will
> be most appreciated!  =)
>
>   Firstly, I was wondering if it would be possible to implement the
> filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
> think this is really not the recommended approach, but an advantage I see
> is it could be used to mount the root filesystem before calling init (that
> would be before user-space exists, right?).  On the other hand, separating
> the work to have a program/daemon in user-space do the communication and
> processing would allow me to write that part in C++ (which I personally
> prefer).
>
>   Secondly, I wonder how I can "bind" a user-space program/daemon with the
> kernel-space of the module.
>
>   Using procfs:  It seems technically possible to achieve my goal with it,
> but I feel this would not be the standard approach.  I would rather keep it
> for live config & live status reporting.
>   Using sysfs:  I've read too much misleading information that I'm not
> sure anymore what it's for or even if it's still used or deprecated!  Can
> someone clarify?  It looks technically identical to procfs, but it's
> mission is a different one, correct?
>   Using pipes:  This may be a good way, but again, I'm not sure if it's
> the standard way.
>   Using IPC:  As far as I understand, IPC would be the way if we'd be
> talking of two threads within the same process, or at least within
> kernel-space.  Though I am likely to be wrong.
>   Using a char device:  It seems technically possible as well, but it may
> be difficult to deal with if I ever have more than one user-space process.
>   Using a block device:  My file system works with files and their
> metadata, but not with blocks, so this is not suitable.  Though it might be
> a nice experiment to build a network block device containing an ext2
> filesystem (which I think would be similar to iSCSI).
>   Using net interface: Not applicable.  And an experiment to try to make
> it applicable doesn't even seem remotely fun.  ;)
>
>   I've seen the functions copy_to/from_user() or something like that, but
> I haven't seen the user-space counter-part of these.  I think this may
> actually be the best approach by far, but I really lack info/docs to go
> that way.  The truly best way might be to use shared memory to avoid
> copying data internally, but I haven't looked into that much yet.
>
>   Finally, is it possible/correct/standard to do something similar to the
> following for the local kernel & user-space communication?  (I haven't
> reviewed the syntax, so take it as some kind of pseudo-code!)
>
> struct foo {...} outgoingFoo;
> // foo contains no pointers, so it is of a fixed size always.
> fwrite(&outgoingFoo, 1, sizeof(outgoingFoo), output);
>   [...then on the other end...]
> if(sizeof(fread( buffer, 1, buffer_size, input);
> struct foo * incomingFoo = (struct foo *) buffer;
>
> Regards,
>   Simon
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130110/87612990/attachment.html 

Kernel-space flirting with user-space

[Simon]

Hi Henrique,
  I browsed the two links and I think I've read parts of the LDD3, but I
could find an answer to my questions.  But considering it is sold as a real
book, I might prefer to buy it.  I find it easier to read in a book than on
screen, for large texts at least.

  And the second link with the howto is seriously impressive!  In fact, if
I had found it earlier, I don't think I would have needed to write my OP!
 :)

Thank you very much,
  Simon

On Thu, Jan 10, 2013 at 10:05 PM, Henrique Rodrigues <
henriquesilvar@gmail.com> wrote:

> I think the LDD book is a good starting point: http://lwn.net/Kernel/LDD3/
>
> You can find some examples of kernel/user communication here:
> http://people.ee.ethz.ch/~arkeller/linux/kernel_user_space_howto.html
>
> --
> Henrique Rodrigues
> http://www.dcc.ufmg.br/~hsr
>
> On Thu, Jan 10, 2013 at 1:09 PM, Simon <turner25@gmail.com> wrote:
>
>> Hi there,
>>
>>   I'm rather new to kernel development, but I am somewhat experienced
>> with Linux, kernel compilation, etc.  I'm just passed writing a helloworld
>> module, dummy filesystem (ramfs wrapper) and proc entry.  I would like to
>> know how communication between kernel & user-space is done (the standard)
>> for a high number of tiny and large messages going in both directions
>> (large messages would need to be transfered incrementally, in chunks, can
>> be sequential or random access).  I plan on developping a network
>> filesystem.  I would appreciate if someone could clarify where I'm wrong
>> and confirm where I'm right, and ideally suggest some vocabulary, source
>> files to lookup, online docs or even books.  The minimum you can give will
>> be most appreciated!  =)
>>
>>   Firstly, I was wondering if it would be possible to implement the
>> filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
>> think this is really not the recommended approach, but an advantage I see
>> is it could be used to mount the root filesystem before calling init (that
>> would be before user-space exists, right?).  On the other hand, separating
>> the work to have a program/daemon in user-space do the communication and
>> processing would allow me to write that part in C++ (which I personally
>> prefer).
>>
>>   Secondly, I wonder how I can "bind" a user-space program/daemon with
>> the kernel-space of the module.
>>
>>   Using procfs:  It seems technically possible to achieve my goal with
>> it, but I feel this would not be the standard approach.  I would rather
>> keep it for live config & live status reporting.
>>   Using sysfs:  I've read too much misleading information that I'm not
>> sure anymore what it's for or even if it's still used or deprecated!  Can
>> someone clarify?  It looks technically identical to procfs, but it's
>> mission is a different one, correct?
>>   Using pipes:  This may be a good way, but again, I'm not sure if it's
>> the standard way.
>>   Using IPC:  As far as I understand, IPC would be the way if we'd be
>> talking of two threads within the same process, or at least within
>> kernel-space.  Though I am likely to be wrong.
>>   Using a char device:  It seems technically possible as well, but it may
>> be difficult to deal with if I ever have more than one user-space process.
>>   Using a block device:  My file system works with files and their
>> metadata, but not with blocks, so this is not suitable.  Though it might be
>> a nice experiment to build a network block device containing an ext2
>> filesystem (which I think would be similar to iSCSI).
>>   Using net interface: Not applicable.  And an experiment to try to make
>> it applicable doesn't even seem remotely fun.  ;)
>>
>>   I've seen the functions copy_to/from_user() or something like that, but
>> I haven't seen the user-space counter-part of these.  I think this may
>> actually be the best approach by far, but I really lack info/docs to go
>> that way.  The truly best way might be to use shared memory to avoid
>> copying data internally, but I haven't looked into that much yet.
>>
>>   Finally, is it possible/correct/standard to do something similar to the
>> following for the local kernel & user-space communication?  (I haven't
>> reviewed the syntax, so take it as some kind of pseudo-code!)
>>
>> struct foo {...} outgoingFoo;
>> // foo contains no pointers, so it is of a fixed size always.
>> fwrite(&outgoingFoo, 1, sizeof(outgoingFoo), output);
>>   [...then on the other end...]
>> if(sizeof(fread( buffer, 1, buffer_size, input);
>> struct foo * incomingFoo = (struct foo *) buffer;
>>
>> Regards,
>>   Simon
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130116/b34d830b/attachment.html 

Kernel-space flirting with user-space

[Kristof Provost]

On 2013-01-10 16:09:49 (-0500), Simon <turner25@gmail.com> wrote:
>   Firstly, I was wondering if it would be possible to implement the
> filesystem entirely as a kernel module.  I would need TCP/UDP sockets.  I
> think this is really not the recommended approach, but an advantage I see
> is it could be used to mount the root filesystem before calling init (that
> would be before user-space exists, right?).  On the other hand, separating
> the work to have a program/daemon in user-space do the communication and
> processing would allow me to write that part in C++ (which I personally
> prefer).

It's certainly possible to implement a network file system entirely in
kernel space. That's where NFS is implemented.
If you *really* care about performance it's pretty much the only way to
go. If you don't you should keep as much as possible in user space.

In either case I'd recommend you start your development on top of fuse
though. That means you can implement all new code in user space, in
whatever language you prefer. It'll be much easier to debug in any case.
Think of it as a fast prototype, to validate your protocol.

Kristof

Kernel-space flirting with user-space

[Simon]

On Fri, Jan 11, 2013 at 10:32 AM, Kristof Provost <kristof@sigsegv.be> wrote:
>
> On 2013-01-10 16:09:49 (-0500), Simon <turner25@gmail.com> wrote:
> >   Firstly, I was wondering if it would be possible to implement the
> > filesystem entirely as a kernel module.  I would need TCP/UDP sockets.
> > I
> > think this is really not the recommended approach, but an advantage I
> > see
> > is it could be used to mount the root filesystem before calling init
> > (that
> > would be before user-space exists, right?).  On the other hand,
> > separating
> > the work to have a program/daemon in user-space do the communication and
> > processing would allow me to write that part in C++ (which I personally
> > prefer).
>
> It's certainly possible to implement a network file system entirely in
> kernel space. That's where NFS is implemented.
> If you *really* care about performance it's pretty much the only way to
> go. If you don't you should keep as much as possible in user space.
>
> In either case I'd recommend you start your development on top of fuse
> though. That means you can implement all new code in user space, in
> whatever language you prefer. It'll be much easier to debug in any case.
> Think of it as a fast prototype, to validate your protocol.

Hi Kristof,
  thanks for your reply.  On my personnal projects, when starting from
scratch, I usually set the goal to "refuse optimization" the code or
the goal to "write inefficient code".  This is mostly a psychological
barrier to avoid premature optimization, which I am prone to.
However, efficiency is a goal for the later stages and so writing the
whole thing in kernel space from the start might be a good idea (less
stuff to rewrite or to "fit in").

  About using FUSE.  I have to say your arguments for its use are
compelling.  However, one of my hidden goals is to learn and practice
kernel programming and if I'd use fuse, I'd be constrained to "fuse
programming" which is also restricted to filesystems.  The day I want
to build something else I would have to start from square one with
kernel programming.

  Also, another project I have in mind after this one is to implement
the same concept, but using a block device instead (I think this comes
very close to what iSCSI does).  It would enable the use of RAID or
LVM on top of that networked block device.  I don't think this will be
more efficient than my current project (specially for use on the
internet), but it might be useful in other ways (for backups perhaps,
or a raid mirror of local device with networked device tuned with
--write-mostly and --write-behind).  At least, mdadm can be used to
handle disconnects/failures, to resync the devices, to manage what
remote hosts are used.  Anyway, this one is still a very foggy idea in
my mind; it has the word "perhaps" stamped in red across it! ;)

  I'll weight the pros and cons for using fuse when I feel ready to
start development.

Thanks again,
  Simon