[ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Matt Ranon]

The Jem team is pleased to announce the release of Kcli, an in-kernel command line interface. Kcli is intended for a special class of embedded Linux applications. The Linux kernel has become the defacto standard OS for embedded applications. This means that Linux is getting bent in some ways that may appear strange to some. One of these ways, is embedded applications that do not use user space. User space consists of a statically linked, one line program, that simply sleeps forever, transforming Linux into a classical embedded RTOS. VxWorks developers will understand what we are talking about, and they may recall how much they depend on the VxWorks shell. Kcli attempts to meet the need for a shell for this class of embedded Linux applications. 
 
Kcli provides a command line environment that runs in the kernel, and that can be extended with custom commands registered by other kernel modules. We have found Kcli invaluable for our development, and we are releasing the patch, in case others find it useful.
 
Kcli is directly derived from libcli written by David Parrish and Brendan O'Dea, and the regular expression support is directly derived from diet libc written by Felix von Leitner.
 
The Jem team fully understands that this kind of patch may not be appropriate for inclusion in the mainline kernel code. We have no expectation that it will be, and we leave that decision fully in the hands of those responsible. Nonetheless, we feel that others may find it useful, and we will also appreciate any appropriate feedback from the community.
 
Kcli is standalone, and modifies no kernel files, except for the Kconfig and Makefile modifications required to wire it into the configuration and build.
 
The Kcli patch is available at ftp://www.javadevices.com/kcli/kcli-v1_0_0.patch.
 
MRanon

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[H. Peter Anvin]

Matt Ranon wrote:
> The Jem team is pleased to announce the release of Kcli, an in-kernel command line interface. Kcli is intended for a special class of embedded Linux applications. The Linux kernel has become the defacto standard OS for embedded applications. This means that Linux is getting bent in some ways that may appear strange to some. One of these ways, is embedded applications that do not use user space. User space consists of a statically linked, one line program, that simply sleeps forever, transforming Linux into a classical embedded RTOS. VxWorks developers will understand what we are talking about, and they may recall how much they depend on the VxWorks shell. Kcli attempts to meet the need for a shell for this class of embedded Linux applications. 

We already have that, it's called initramfs; you can use either busybox 
or klibc ash in this space, and no need to run it in kernel space.

Running this kind of crap in kernel space is lunacy in the extreme.

	-hpa

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Andrew Morton]

On Mon, 23 Apr 2007 14:31:39 -0700 (PDT)
Matt Ranon <mranon@javadevices.com> wrote:
>

(text reformatted to less than 80 cols.  Please, we'll get along a lot
better if you don't send 1000-column emails)

> The Jem team is pleased to announce the release of Kcli, an in-kernel
> command line interface.  Kcli is intended for a special class of embedded
> Linux applications.  The Linux kernel has become the defacto standard OS
> for embedded applications.  This means that Linux is getting bent in some
> ways that may appear strange to some.  One of these ways, is embedded
> applications that do not use user space.  User space consists of a
> statically linked, one line program, that simply sleeps forever,
> transforming Linux into a classical embedded RTOS.  VxWorks developers will
> understand what we are talking about, and they may recall how much they
> depend on the VxWorks shell.  Kcli attempts to meet the need for a shell
> for this class of embedded Linux applications.  

Alas, we are not vxworks developers, and probably most of us know zero
about the use-cases for this feature, why embedded systems find it
valuable, etc.

So it's up to you to tell us all this.

> Kcli provides a command line environment that runs in the kernel, and that
> can be extended with custom commands registered by other kernel modules. 
> We have found Kcli invaluable for our development, and we are releasing the
> patch, in case others find it useful.
> 
> Kcli is directly derived from libcli written by David Parrish and Brendan
> O'Dea, and the regular expression support is directly derived from diet
> libc written by Felix von Leitner.
> 
> The Jem team fully understands that this kind of patch may not be
> appropriate for inclusion in the mainline kernel code.  We have no
> expectation that it will be, and we leave that decision fully in the hands
> of those responsible.

We don't have enough information to make that call.

>  Nonetheless, we feel that others may find it useful,
> and we will also appreciate any appropriate feedback from the community.
> 
> Kcli is standalone, and modifies no kernel files, except for the Kconfig
> and Makefile modifications required to wire it into the configuration and
> build.

The obvious question is: what's _wrong_ with doing all this in some
cut-down userspace environment like busybox?  Why is this stuff better?

Obviously some embedded developers have considered that option and
have rejected it.  But we do need to be told, at length, why that
decision was made.

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Matt Ranon]

> (text reformatted to less than 80 cols.  Please, we'll get along a lot
> better if you don't send 1000-column emails)

Sorry. I am afraid we are from a different background, and so very
poorly versed in these things. My email client does not seem
to have an option to tell it to format in 80 cols. So, hopefully,
using CR, I am achieving the same effect. Let me know if
it doesn't work, and I will have to switch to a different email
client for conversing with the lkml.


> The obvious question is: what's _wrong_ with doing all this in some
> cut-down userspace environment like busybox?  Why is this stuff better?
> 
> Obviously some embedded developers have considered that option and
> have rejected it.  But we do need to be told, at length, why that
> decision was made.

There is nothing _wrong_ with doing it all in a cut-down userspace. It
is a matter of personal preference, culture, and the application. That
is what makes Linux so great, it is all about choice.

We are developing devices that don't have a user space, and we don't
see the point in including one just for debug purposes. We will not be 
offended if Kcli is not included into the kernel mainline, nor if Kcli compels
people to call us stupid (as it already has) just because we are different 
and some people don't understand us. We are firm believers that the 
world, including the Linux kernel world,  would be a nasty place if there 
was only _one_ way to do any given task. Additionally, we  are almost 
certain that there will be others who think like we do, so we are reaching 
out to them. We also feel compelled to give _something_ back to the 
community that has given so much to us, and, for now, this is all we have.

However, our reasons for Kcli are:
1) Our devices ship with no user space, and we want the development
environment to be as close as possible to the final product.
2) Getting debug information with user space calls require context 
switches and data copies, which changes the real time profile and can mask
bugs. 
3) To use user space, we would need cross compiled libc's, special
builds of gcc, root file systems, flash storage to store it all, and all 
sorts of things which make life a lot more complicated than it needs 
to be for us. We are quite capable of producing all these things, but,
we just don't see the point in it. Our way, we just have a gcc capable 
of cross compiling the kernel and it is so simple.
4) For us, it is the opposite argument. We would need to be convinced
that having user space is worth all the overhead. Not just CPU
overhead, but all the overheads.
5) We like it in the kernel, we find it to be warm and fuzzy. Whereas,
user space is a cold, dark, and rainy place, and we just don't want to
go there. :)

We do not claim to have come up with a _better_ way. We have just
created something that we feel would be useful to others.

MRanon.

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Satyam Sharma]

Hi Matt,

On 4/24/07, Matt Ranon <mranon@javadevices.com> wrote:
> > The obvious question is: what's _wrong_ with doing all this in some
> > cut-down userspace environment like busybox?  Why is this stuff better?
> >
> > Obviously some embedded developers have considered that option and
> > have rejected it.  But we do need to be told, at length, why that
> > decision was made.
>
> There is nothing _wrong_ with doing it all in a cut-down userspace. It
> is a matter of personal preference, culture, and the application. That
> is what makes Linux so great, it is all about choice.
>
> We are developing devices that don't have a user space, and we don't
> see the point in including one just for debug purposes. We will not be
> offended if Kcli is not included into the kernel mainline, nor if Kcli compels
> people to call us stupid (as it already has) just because we are different
> and some people don't understand us. We are firm believers that the
> world, including the Linux kernel world,  would be a nasty place if there
> was only _one_ way to do any given task. Additionally, we  are almost
> certain that there will be others who think like we do, so we are reaching
> out to them. We also feel compelled to give _something_ back to the
> community that has given so much to us, and, for now, this is all we have.

I'm afraid you might've misunderstood the (rather caustic, sometimes)
general nature of comments on lkml :-) But I guess you only have
everything to gain if you use features that have been developed (and
are being *maintained* in the current kernel) that already do the kind
of stuff you want done.

You might have your reasons for being so anxious to avoid any
userspace at all, but quoting famous words, continuing to maintain
Kcli out-of-tree could soon turn out to be an act for
self-flagellation for you :-)

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

Satyam Sharma schrieb:
> You might have your reasons for being so anxious to avoid any
> userspace at all, but quoting famous words, continuing to maintain
> Kcli out-of-tree could soon turn out to be an act for
> self-flagellation for you :-)

OTOH, considering certain recent comments on this list[1],
maintaining it in-tree might turn out to be not that much
better.

SCNR
T.

[1] <20070422172017.52f21a90@the-village.bc.nu> or http://lkml.org/lkml/2007/4/22/139

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Sam Ravnborg]

On Tue, Apr 24, 2007 at 02:36:48PM +0200, Tilman Schmidt wrote:
> Satyam Sharma schrieb:
> > You might have your reasons for being so anxious to avoid any
> > userspace at all, but quoting famous words, continuing to maintain
> > Kcli out-of-tree could soon turn out to be an act for
> > self-flagellation for you :-)
> 
> OTOH, considering certain recent comments on this list[1],
> maintaining it in-tree might turn out to be not that much
> better.
> 
> SCNR
> T.
> 
> [1] <20070422172017.52f21a90@the-village.bc.nu> or http://lkml.org/lkml/2007/4/22/139
Trolling?
This is comparing apples and oranges. There is no replacemnt in the works for kcli and
the complexity of the code idffers a lot.
I would say that readability differ a lot too.

	Sam

Re: [ANNOUNCE][PATCH] Kcli - Kernel command line interface.

[Matt Mackall]

On Mon, Apr 23, 2007 at 06:12:19PM -0700, Matt Ranon wrote:
> However, our reasons for Kcli are:
> 1) Our devices ship with no user space, and we want the development
> environment to be as close as possible to the final product.

I hope that means your devices have full source code available under
the GPL. Even if they do, kcli will probably encourage other folks to
abuse the kernel license.

> 2) Getting debug information with user space calls require context 
> switches and data copies, which changes the real time profile and can mask
> bugs.

Probably no worse than whatever I/O interface you're likely to use
with this. Reminds me of a particular "world-famous filesystem
developer" who eventually figured out that his serial console was the
cause of most of the filesystem latency glitches he was hitting.

> 3) To use user space, we would need cross compiled libc's, special
> builds of gcc, root file systems, flash storage to store it all, and all 
> sorts of things which make life a lot more complicated than it needs 
> to be for us. We are quite capable of producing all these things, but,
> we just don't see the point in it. Our way, we just have a gcc capable 
> of cross compiling the kernel and it is so simple.

Please familiarize yourself with initramfs and klibc. You can attach
an arbitrary minimal filesystem to the kernel image. You don't need a
special compiler, flash, etc., and the kernel build system will build
the filesystem image (a cpio archive) for you.

> 4) For us, it is the opposite argument. We would need to be convinced
> that having user space is worth all the overhead. Not just CPU
> overhead, but all the overheads.

Moving stuff to userspace usually has negative overhead because it's
pageable, easier to maintain, and easier to debug.

> 5) We like it in the kernel, we find it to be warm and fuzzy. Whereas,
> user space is a cold, dark, and rainy place, and we just don't want to
> go there. :)

I once spent a few weeks getting OpenSSH serving multiple clients on
vxWorks and I'm happy I'll never have to touch a kernel without
userspace again.

I think there probably is a place for some basic debugging and
introspection, including dumping address ranges, looking up symbols
and the like, but a debugger is different than a command interpreter.

-- 
Mathematics is the supreme nostalgia of our time.