klibc development release

klibc development release

[H. Peter Anvin]

Okay, I'm starting to have enough guts about this to release for
testing...

klibc is a tiny C library subset intended to be integrated into the
kernel source tree and being used for initramfs stuff.  Thus,
initramfs+rootfs can be used to move things that are currently in
kernel space, such as ip autoconfiguration or nfsroot (in fact,
mounting root in general) into user space.

I would particularly appreciate portability comments, since I'm flying
blind on non-i386 machines (anyone want to send me hardware?),
although any bug reports would be appreciated.

    ftp://ftp.kernel.org/pub/linux/libs/klibc/klibc.tar.gz

I haven't bothered putting a version number on it, since it changes
quite often.  I have also published the CVS repository in the
directory above.

	-hpa


P.S. I'm aware gregkh already started a klibc project; I have
discussed it with him and we have agreed my project supercedes his.

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: klibc development release

[Andreas Schwab]

"H. Peter Anvin" <hpa@zytor.com> writes:

|> I would particularly appreciate portability comments, since I'm flying
|> blind on non-i386 machines (anyone want to send me hardware?),

You don't need hardware, just use a cross-compiler.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: klibc development release

[H. Peter Anvin]

Andreas Schwab wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
> |> I would particularly appreciate portability comments, since I'm flying
> |> blind on non-i386 machines (anyone want to send me hardware?),
> 
> You don't need hardware, just use a cross-compiler.
> 

I'd like to actually be able to test something.  The cross-compiler 
helps somewhat, of course, but doesn't let me test anything.

	-hpa

Re: klibc development release

[Arnd Bergmann]

H. Peter Anvin wrote:

> I would particularly appreciate portability comments, since I'm flying
> blind on non-i386 machines (anyone want to send me hardware?),
> although any bug reports would be appreciated.

I just tried compiling for 64 bit s390x, this is what I found so far:

- in include/stdint.h, the definitions for *int_fast*_t conflict with
  the 64 bit ones

- mmap.c does not compile because _syscall6 is defined only for i386,
  alpha, cris, mips(64) and parisc but not for the other ten architectures.

- syscalls/time.o does not compile because __NR_time is not defined for
  alpha, ia64 and s390x

- making klibc cross compilable with 'make CROSS_COMPILE=prefix-' would be 
  a nice and trivial addition

- compile time warnings: see below

more later when I actually manage to run something

	Arnd <><

testvsnp.c: In function `main':
testvsnp.c:14: warning: int format, different type arg (arg 3)
testvsnp.c:17: warning: int format, different type arg (arg 3)
testvsnp.c:20: warning: int format, different type arg (arg 3)
testvsnp.c:23: warning: int format, different type arg (arg 3)
testvsnp.c:31: warning: int format, different type arg (arg 3)
testvsnp.c:36: warning: int format, different type arg (arg 3)
testvsnp.c:39: warning: int format, different type arg (arg 3)
testvsnp.c:42: warning: int format, different type arg (arg 3)
testvsnp.c:45: warning: int format, different type arg (arg 3)
testvsnp.c:48: warning: int format, different type arg (arg 3)
testvsnp.c:51: warning: int format, different type arg (arg 3)
testvsnp.c:54: warning: int format, different type arg (arg 3)
testvsnp.c:57: warning: int format, different type arg (arg 3)
testvsnp.c:60: warning: int format, different type arg (arg 3)
testvsnp.c:63: warning: int format, different type arg (arg 3)
testvsnp.c:66: warning: int format, different type arg (arg 3)
testvsnp.c:69: warning: int format, different type arg (arg 3)
testvsnp.c:72: warning: int format, different type arg (arg 3)
testvsnp.c:75: warning: int format, different type arg (arg 3)
testvsnp.c:79: warning: int format, different type arg (arg 3)
testvsnp.c:83: warning: int format, different type arg (arg 3)
testvsnp.c:87: warning: int format, different type arg (arg 3)
testvsnp.c:90: warning: int format, different type arg (arg 3)
testvsnp.c:93: warning: int format, different type arg (arg 3)
testvsnp.c:96: warning: int format, different type arg (arg 3)
testvsnp.c:99: warning: int format, different type arg (arg 3)
testvsnp.c:102: warning: int format, different type arg (arg 3)
testvsnp.c:105: warning: int format, different type arg (arg 3)
testvsnp.c:108: warning: int format, different type arg (arg 3)
testvsnp.c:111: warning: int format, different type arg (arg 3)
vfprintf.c: In function `vfprintf':
vfprintf.c:26: warning: cast from pointer to integer of different size
fputs.c: In function `fputs':
fputs.c:15: warning: cast from pointer to integer of different size
calloc.c: In function `calloc':
calloc.c:16: warning: type mismatch in implicit declaration for built-in function `memset'
vfprintf.c: In function `vfprintf':
vfprintf.c:26: warning: cast from pointer to integer of different size
fputs.c: In function `fputs':
fputs.c:15: warning: cast from pointer to integer of different size
calloc.c: In function `calloc':

Re: klibc development release

[Arnd Bergmann]

Arnd Bergmann wrote:
> more later when I actually manage to run something

some regression data from running the test programs on s390 and s390x:

test		31 bit			64 bit
-----------------------------------------------------
microhello	ok			ok
minihello	ok			ok
hello		ok			ok
getenvtest	wrong output		segfault in strncmp
getopttest	wrong output (getenv) segfault from getenv/strncmp
malloctest	mmap not implemented 	mmap not implemented
testvsnp	ok			wrong output

You could say you are 50% there ;-)

For the autogenerated system call objects, a quick grep in include/asm-*/unistd
showed some more that appear to be missing on some platforms:

alpha: alarm, fdatasync, getegid, geteuid, getgid, getpid, getppid, getuid, nice,
	pause, umount2, utime, time
ia64:	alarm, fork, getpgrp, nice, pause, umount2, utime, vfork, time
mips64: pivot_root, select, vfork
mips:	vfork
parisc: select, umount
x86_64:	nice, umount
sparc: setresgid, setresuid
s390x: time
s390: select
m68k: pivot_root

Re: klibc development release

[H. Peter Anvin]

Arnd Bergmann wrote:
>> You could say you are 50% there ;-)
> 
> For the autogenerated system call objects, a quick grep in include/asm-*/unistd
> showed some more that appear to be missing on some platforms:
> 
> alpha: alarm, fdatasync, getegid, geteuid, getgid, getpid, getppid, getuid, nice,
> 	pause, umount2, utime, time
> ia64:	alarm, fork, getpgrp, nice, pause, umount2, utime, vfork, time
> mips64: pivot_root, select, vfork
> mips:	vfork
> parisc: select, umount
> x86_64:	nice, umount
> sparc: setresgid, setresuid
> s390x: time
> s390: select
> m68k: pivot_root
 >

Hmf... some of these seem to be outright omissions (pivot_root() and 
umount2() especially), and probably indicate bugs or that the stock 
kernel isn't up to date anymore.

I can see umount() being missing (as in "use umount2()").  time() I 
guess can be emulated with gettimeofday(), but select() being missing is 
rather odd (is this a "emulate using poll()" philosophy?  Barf...)

If the maintainers of the above ports could give me a pointer to how the 
above system calls are expected to be done I'd appreciate it.  I'll 
probably look through the glibc sources for hints in the meantime.

	-hpa

Re: klibc development release

[David Mosberger]

>>>>> On Fri, 09 Aug 2002 11:55:20 -0700, "H. Peter Anvin" <hpa@zytor.com> said:

  HPA> Hmf... some of these seem to be outright omissions
  HPA> (pivot_root() and umount2() especially), and probably indicate
  HPA> bugs or that the stock kernel isn't up to date anymore.

  HPA> I can see umount() being missing (as in "use umount2()").

Alpha calls umount2() "oldumount"; ia64 never had a one-argument
version of umount(), so there is no point creating legacy (and the
naming is inconsistent anyhow...).

	--david

Re: klibc development release

[H. Peter Anvin]

David Mosberger wrote:
>>>>>>On Fri, 09 Aug 2002 11:55:20 -0700, "H. Peter Anvin" <hpa@zytor.com> said:
>>>>>
> 
>   HPA> Hmf... some of these seem to be outright omissions
>   HPA> (pivot_root() and umount2() especially), and probably indicate
>   HPA> bugs or that the stock kernel isn't up to date anymore.
> 
>   HPA> I can see umount() being missing (as in "use umount2()").
> 
> Alpha calls umount2() "oldumount"; ia64 never had a one-argument
> version of umount(), so there is no point creating legacy (and the
> naming is inconsistent anyhow...).
> 

The gratuitous inconsistencies between platforms is something that is 
currently driving me up the wall.  I'm starting to think the NetBSD 
people have the right idea: when you add a system call on NetBSD, you 
only have to add it in one place and it becomes available on all the 
platforms they support.  Of course, you can provide a custom 
implementation for any one platform, but the idea is to keep as much of 
the code generic as possible...

	-hpa

Re: klibc development release

[Matti Aarnio]

On Fri, Aug 09, 2002 at 12:14:00PM -0700, H. Peter Anvin wrote:
> David Mosberger wrote:
...
> >Alpha calls umount2() "oldumount"; ia64 never had a one-argument
> >version of umount(), so there is no point creating legacy (and the
> >naming is inconsistent anyhow...).
> 
> The gratuitous inconsistencies between platforms is something that is 
> currently driving me up the wall.  I'm starting to think the NetBSD 
> people have the right idea: when you add a system call on NetBSD, you 
> only have to add it in one place and it becomes available on all the 
> platforms they support.

  History...   Initially I thought you are describing something
  different from Linux model (after all, different platforms have
  different ABIs for syscalls, leading to different call-vector
  tables, etc.)  but no, I see no difference from this description.

  How NetBSD handles the issue, I don't know.   One interpretation
  of what you say is that when a new architecture is added to NetBSD,
  it will instantly inherit the entire historical set of syscalls,
  including the obsolete ones.

  As long as old calls are supported, the kernel needs to know
  how to handle a call serving some thing 10 years ago.. ( e.g.
  "oldumount").  Ok, some have been revoked, but not everything
  of the very old and obsolete stuff.


  How this reflects to  klibc ?     What I understand of klibc,
  it can track very closely the kernel in question.  Introducing
  new improved syscall to do XYZ obsoliting ABC, whimblam, you
  replace it.  No need to carry code supporting any other version
  of kernel than for what the stuff is made for.
  ("Small is beautifull", and all that...)


  glibc folks are (to an extreme pain) supporting kernels 2.0
  (if not from before) to current bleeding edge, and emulating
  all fancy dancing available with new system services by doing
  some weird gimmicks..


>     Of course, you can provide a custom implementation for
> any one platform, but the idea is to keep as much of the code
> generic as possible...

  Desire to support running of "native UNIX" binaries means
  emulating their syscalls.  Initially Linux ran MINIX binaries,
  then came SCO binaries, and Alpha running some of OSF/1 binaries...

> 	-hpa

/Matti Aarnio

Re: klibc development release

[H. Peter Anvin]

Matti Aarnio wrote:
> 
>   History...   Initially I thought you are describing something
>   different from Linux model (after all, different platforms have
>   different ABIs for syscalls, leading to different call-vector
>   tables, etc.)  but no, I see no difference from this description.
> 
>   How NetBSD handles the issue, I don't know.   One interpretation
>   of what you say is that when a new architecture is added to NetBSD,
>   it will instantly inherit the entire historical set of syscalls,
>   including the obsolete ones.
> 

What NetBSD does is that they treat the system call mechanism as a
special kind of function call... like an RPC stub kind of thing.  Just
like it's not visible to the C programmer how a particular architecture
is implementing a function call, on NetBSD all one really needs to know
is the system call number (which is architecture independent!) and its
prototype, and you can generate both user-side and kernel-side stubs for
this system call, ending up with it calling a function in the generic
kernel code (sys_mmap(), or whatever.)

It seems to me that the big losers of the Linux model are the smaller
architectures, since most people aren't going to be able to know when
they're breaking something on, for example, alpha, cris or s390, and
they're certainly not going to know how to add something to them.

	-hpa

Re: klibc development release

[Erik Andersen]

On Sat Aug 10, 2002 at 01:27:36AM +0300, Matti Aarnio wrote:
>   glibc folks are (to an extreme pain) supporting kernels 2.0
>   (if not from before) to current bleeding edge, and emulating
>   all fancy dancing available with new system services by doing
>   some weird gimmicks..

The glibc folks care about binary compatibility.  klibc doesn't
need to care about such things, and therefore need not care about
such baggage.

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: klibc development release

[H. Peter Anvin]

Erik Andersen wrote:
> On Sat Aug 10, 2002 at 01:27:36AM +0300, Matti Aarnio wrote:
> 
>>  glibc folks are (to an extreme pain) supporting kernels 2.0
>>  (if not from before) to current bleeding edge, and emulating
>>  all fancy dancing available with new system services by doing
>>  some weird gimmicks..
> 
> The glibc folks care about binary compatibility.  klibc doesn't
> need to care about such things, and therefore need not care about
> such baggage.
> 

Exactly.

	-hpa

Re: klibc development release

[Christoph Hellwig]

On Sat, Aug 10, 2002 at 01:27:36AM +0300, Matti Aarnio wrote:
>   How NetBSD handles the issue, I don't know.   One interpretation
>   of what you say is that when a new architecture is added to NetBSD,
>   it will instantly inherit the entire historical set of syscalls,
>   including the obsolete ones.

netbsd puts all syscall code not needed by the current release under a
per-version ifdef.  A new port starting at, say 1.4, will never have
this enabled (unless it has binary emulations that need parts of the
old syscalls)

Re: klibc development release

[H. Peter Anvin]

Christoph Hellwig wrote:
> On Sat, Aug 10, 2002 at 01:27:36AM +0300, Matti Aarnio wrote:
> 
>>  How NetBSD handles the issue, I don't know.   One interpretation
>>  of what you say is that when a new architecture is added to NetBSD,
>>  it will instantly inherit the entire historical set of syscalls,
>>  including the obsolete ones.
> 
> netbsd puts all syscall code not needed by the current release under a
> per-version ifdef.  A new port starting at, say 1.4, will never have
> this enabled (unless it has binary emulations that need parts of the
> old syscalls)

Sure, but that still means the *current* ABI is consistent between 
platforms.

	-hpa

Re: klibc development release

[Christoph Hellwig]

On Sat, Aug 10, 2002 at 03:48:09AM -0700, H. Peter Anvin wrote:
> Sure, but that still means the *current* ABI is consistent between 
> platforms.

yes, that's why I like the way they do it.  It doesn't really impose any
overhead and is much, much cleaner.

Re: klibc development release

[H. Peter Anvin]

Christoph Hellwig wrote:
> On Sat, Aug 10, 2002 at 03:48:09AM -0700, H. Peter Anvin wrote:
> 
>>Sure, but that still means the *current* ABI is consistent between 
>>platforms.
> 
> yes, that's why I like the way they do it.  It doesn't really impose any
> overhead and is much, much cleaner.

100% in agreement.

	-hpa

Re: klibc development release

[Erik Andersen]

On Fri Aug 09, 2002 at 12:14:00PM -0700, H. Peter Anvin wrote:
> David Mosberger wrote:
> >>>>>>On Fri, 09 Aug 2002 11:55:20 -0700, "H. Peter Anvin" <hpa@zytor.com> 
> >>>>>>said:
> >>>>>
> >
> >  HPA> Hmf... some of these seem to be outright omissions
> >  HPA> (pivot_root() and umount2() especially), and probably indicate
> >  HPA> bugs or that the stock kernel isn't up to date anymore.
> >
> >  HPA> I can see umount() being missing (as in "use umount2()").
> >
> >Alpha calls umount2() "oldumount"; ia64 never had a one-argument
> >version of umount(), so there is no point creating legacy (and the
> >naming is inconsistent anyhow...).
> >
> 
> The gratuitous inconsistencies between platforms is something that is 
> currently driving me up the wall.  I'm starting to think the NetBSD 
> people have the right idea: when you add a system call on NetBSD, you 
> only have to add it in one place and it becomes available on all the 
> platforms they support.  Of course, you can provide a custom 
> implementation for any one platform, but the idea is to keep as much of 
> the code generic as possible...

Amen brother.   That would be great!  But I'm not holding 
my breath waiting to see it,

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: klibc development release

[H. Peter Anvin]

David Mosberger wrote:
>>>>>>On Fri, 09 Aug 2002 11:55:20 -0700, "H. Peter Anvin" <hpa@zytor.com> said:
>>>>>
> 
>   HPA> Hmf... some of these seem to be outright omissions
>   HPA> (pivot_root() and umount2() especially), and probably indicate
>   HPA> bugs or that the stock kernel isn't up to date anymore.
> 
>   HPA> I can see umount() being missing (as in "use umount2()").
> 
> Alpha calls umount2() "oldumount"; ia64 never had a one-argument
> version of umount(), so there is no point creating legacy (and the
> naming is inconsistent anyhow...).
> 

You mean alpha calls umount2() "umount" and umount() "oldumount"?

	-hpa

Re: klibc development release

[David Mosberger]

>>>>> On Fri, 09 Aug 2002 12:55:24 -0700, "H. Peter Anvin" <hpa@zytor.com> said:

  HPA> You mean alpha calls umount2() "umount" and umount()
  HPA> "oldumount"?

Yes.

	--david

Re: klibc development release

[H. Peter Anvin]

David Mosberger wrote:
>>>>>>On Fri, 09 Aug 2002 12:55:24 -0700, "H. Peter Anvin" <hpa@zytor.com> said:
>>>>>
> 
>   HPA> You mean alpha calls umount2() "umount" and umount()
>   HPA> "oldumount"?
> 

Cool; I have added this.  I have uploaded the new version as
klibc-0.1.tar.gz to get the version number thing going...

	-hpa

Re: klibc development release

[Albert D. Cahalan]

> klibc is a tiny C library subset intended to be integrated into the
> kernel source tree and being used for initramfs stuff.  Thus,
> initramfs+rootfs can be used to move things that are currently in
> kernel space, such as ip autoconfiguration or nfsroot (in fact,
> mounting root in general) into user space.

Could I link 4-clause BSD source against this?
(the GPL is incompatible with the 4-clause BSD license)

> I would particularly appreciate portability comments, since I'm flying
> blind on non-i386 machines (anyone want to send me hardware?),
> although any bug reports would be appreciated.
>
>     ftp://ftp.kernel.org/pub/linux/libs/klibc/klibc.tar.gz
>
> I haven't bothered putting a version number on it, since it changes
> quite often.  I have also published the CVS repository in the
> directory above.

I could test on PowerPC, but wouldn't be able to tell you
if I'm testing the latest code or not. You don't need to
get creative with the version number; an integer is fine.

Re: klibc development release

[H. Peter Anvin]

Albert D. Cahalan wrote:
>>klibc is a tiny C library subset intended to be integrated into the
>>kernel source tree and being used for initramfs stuff.  Thus,
>>initramfs+rootfs can be used to move things that are currently in
>>kernel space, such as ip autoconfiguration or nfsroot (in fact,
>>mounting root in general) into user space.
> 
> Could I link 4-clause BSD source against this?
> (the GPL is incompatible with the 4-clause BSD license)
> 

I'm planning to release this under a BSD-like license, such as 3-clause
BSD, MIT or the X license.  I'm still looking at each of those.

> 
>>I would particularly appreciate portability comments, since I'm flying
>>blind on non-i386 machines (anyone want to send me hardware?),
>>although any bug reports would be appreciated.
>>
>>    ftp://ftp.kernel.org/pub/linux/libs/klibc/klibc.tar.gz
>>
>>I haven't bothered putting a version number on it, since it changes
>>quite often.  I have also published the CVS repository in the
>>directory above.
> 
> I could test on PowerPC, but wouldn't be able to tell you
> if I'm testing the latest code or not. You don't need to
> get creative with the version number; an integer is fine.
> 

Good point.

	-hpa

Re: klibc development release

[Rob Landley]

On Friday 09 August 2002 04:22 pm, H. Peter Anvin wrote:
> Albert D. Cahalan wrote:
> >>klibc is a tiny C library subset intended to be integrated into the
> >>kernel source tree and being used for initramfs stuff.  Thus,
> >>initramfs+rootfs can be used to move things that are currently in
> >>kernel space, such as ip autoconfiguration or nfsroot (in fact,
> >>mounting root in general) into user space.
> >
> > Could I link 4-clause BSD source against this?
> > (the GPL is incompatible with the 4-clause BSD license)
>
> I'm planning to release this under a BSD-like license, such as 3-clause
> BSD, MIT or the X license.  I'm still looking at each of those.

What's wrong with LGPL?  I thought libraries were what it was originally 
intended for.  (Is 4 clause BSD incompatable with LGPL?)

Yeah, I know stallman's decided to hate.  I'm sure he'd be happy to know that 
stance encourages stuff to be released BSD-ish instead. :)

Rob

Re: klibc development release

[Alexander Viro]

On Sun, 11 Aug 2002, Rob Landley wrote:

> What's wrong with LGPL?  I thought libraries were what it was originally 

klibc is static-only.  So for all practical purposes LGPL would be every bit
as viral as GPV itself.

Re: klibc development release

[Oliver Xymoron]

On Sun, 11 Aug 2002, Alexander Viro wrote:

> > What's wrong with LGPL?  I thought libraries were what it was originally
>
> klibc is static-only.  So for all practical purposes LGPL would be every bit
> as viral as GPV itself.

You say that as if it were a bad thing.

(And technically incorrect, if you also provide .o files so that the end
user can relink as they desire.)

That aside for the moment, isn't the plan to pull stuff that's currently
GPL out of the kernel and link against this lib anyway?

Second point - if it goes into the kernel source tree as suggested, but
with a 'copycenter' license, we can expect to have the nVidia problem but
worse.

Making it GPL shouldn't be a big deal. It is intended to be a small amount
of code, after all. And I'd hate to get into a situation where people
started shipping their magic 'make the hardware work' bits as closed
replacements for the early boot code.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: klibc development release

[Alexander Viro]

On Sun, 11 Aug 2002, Oliver Xymoron wrote:

> On Sun, 11 Aug 2002, Alexander Viro wrote:
> 
> > > What's wrong with LGPL?  I thought libraries were what it was originally
> >
> > klibc is static-only.  So for all practical purposes LGPL would be every bit
> > as viral as GPV itself.
> 
> You say that as if it were a bad thing.

I do.  I have no problems with people choosing whatever license they prefer
for their work and I have no problems with using GPL when I'm working on
projects that are already under it, but it's not the license I would choose
for my work in cases when I have a choice.

As for the "make the hardware work" code, there's nothing to stop people from
doing that _NOW_.  I'm not too fond of that, but as long as we are talking
about userland code it
	* will have to use normal system calls
	* will not have to link against any particular library, no matter
what we provide
	* will be up to those who write it and those who decide to use it.

We are talking about libc.  _Nothing_ in that code couldn't be reimplemented
by any half-competent programmer.  It's a textbook stuff.  Those who don't
like GPL would be trivially able to reimplement all these functions in their
own code anyway.  End of story.  Whatever license is chosen, it won't prevent
people from putting their code under any license they like.

There is a crucial difference from the situation with nVidia, Veritrash and
the rest of let's-bugger-the-kernel team.  _They_ want more than using
syscalls from user mode - they want an access to guts of the kernel and that's
a very different can of worms.  And _that_ I have problems with.  A lot.
Especially when they expect us to abstain from changes of kernel internals
that might break their junk and when they whine when such changes are done.

People do have a right to put their code under whatever license they like.
Now, _I_ won't use the stuff I don't have a source for unless I have
exceptionally good reason to believe that authors of that stuff are
among the few percents of programmers who *can* find their arse without
outside help.  But that has nothing to do with licensing or any moral
considerations and everything to the fact that I know what kind of crap
most of the software is.

Re: klibc development release

[Oliver Xymoron]

On 8 Aug 2002, H. Peter Anvin wrote:

> Okay, I'm starting to have enough guts about this to release for
> testing...
>
> klibc is a tiny C library subset intended to be integrated into the
> kernel source tree and being used for initramfs stuff.  Thus,
> initramfs+rootfs can be used to move things that are currently in
> kernel space, such as ip autoconfiguration or nfsroot (in fact,
> mounting root in general) into user space.

Remind me why we'd want this in the kernel source tree when we don't even
have modutils there? Or a real bootloader? There is no requirement that
the kernel must be able to build a bootable image with ip autoconf and
nfsroot, etc. without using external tools. A minimal 'parse command line
to mount root and call real init' might be nice, but mostly as an
example, like the example watchdog code.

I think a better way to go is to simply roll an initbootutils.tar.gz,
point to it in the kernel CHANGES, etc., start pulling stuff into it, and
people will catch on in no time.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: klibc development release

[Rob Landley]

On Friday 09 August 2002 06:03 pm, Oliver Xymoron wrote:
> On 8 Aug 2002, H. Peter Anvin wrote:
> > Okay, I'm starting to have enough guts about this to release for
> > testing...
> >
> > klibc is a tiny C library subset intended to be integrated into the
> > kernel source tree and being used for initramfs stuff.  Thus,
> > initramfs+rootfs can be used to move things that are currently in
> > kernel space, such as ip autoconfiguration or nfsroot (in fact,
> > mounting root in general) into user space.
>
> Remind me why we'd want this in the kernel source tree when we don't even
> have modutils there?

It's optional: CONFIG_MODULES=n

> Or a real bootloader?

Also optional: cat bzImage > /dev/fd0 (ugly but still works)

> There is no requirement that
> the kernel must be able to build a bootable image with ip autoconf and
> nfsroot, etc. without using external tools.

How about partition detection?  When initramfs goes in that's one of the 
things they're threatening to move to userspace.  Also lots of the hardware 
detection and setup (ACPI, hotplug style PCI probing...)

If that sort of stuff goes into userspace, you may not be able to boot ANY 
linux system without it.

> A minimal 'parse command line
> to mount root and call real init' might be nice, but mostly as an
> example, like the example watchdog code.

See "partition detection in userspace".  How minimal is "minimal"?

> I think a better way to go is to simply roll an initbootutils.tar.gz,
> point to it in the kernel CHANGES, etc., start pulling stuff into it, and
> people will catch on in no time.

Possibly it could be gotten to work before being split off?  And then there's 
the question of upgrading your kernel without upgrading the initbootutils: if 
this never happens, what exactly is the benefit of it being in another 
tarball?  (I.E. how tight is the binding gonna be?  I certainly don't know 
yet...)

Rob

Re: klibc development release

[Oliver Xymoron]

On Sun, 11 Aug 2002, Rob Landley wrote:

> On Friday 09 August 2002 06:03 pm, Oliver Xymoron wrote:
> > On 8 Aug 2002, H. Peter Anvin wrote:
> > > Okay, I'm starting to have enough guts about this to release for
> > > testing...
> > >
> > > klibc is a tiny C library subset intended to be integrated into the
> > > kernel source tree and being used for initramfs stuff.  Thus,
> > > initramfs+rootfs can be used to move things that are currently in
> > > kernel space, such as ip autoconfiguration or nfsroot (in fact,
> > > mounting root in general) into user space.
> >
> > Remind me why we'd want this in the kernel source tree when we don't even
> > have modutils there?
>
> It's optional: CONFIG_MODULES=n

For some configurations. Part of this boot in userspace deal was more
reliance on modules and late initialization.

> > Or a real bootloader?
>
> Also optional: cat bzImage > /dev/fd0 (ugly but still works)

On some machines. I don't have any with floppies. Making a bootable CDROM
without a real userspace is kindof silly. And I can't do a whole heck of a
lot with that floppy unless I've got some other packages around anyway.

> > There is no requirement that
> > the kernel must be able to build a bootable image with ip autoconf and
> > nfsroot, etc. without using external tools.
>
> How about partition detection?  When initramfs goes in that's one of the
> things they're threatening to move to userspace.  Also lots of the hardware
> detection and setup (ACPI, hotplug style PCI probing...)

Yes, yes. The whole point of this exercise is to move such things out of
the kernel. The mechanics of booting can (and should) be made entirely
independent of the kernel tree. LILO, grub, etc., do this on one side,
bootutils.tgz should do this on the other. The kernel's job here is to get
some form of userspace running and say 'not my job'. Having an 'official'
bootutils tied to the kernel effectively keeps it kernel policy and will
discourage people from rolling their own alternatives.

> If that sort of stuff goes into userspace, you may not be able to boot ANY
> linux system without it.

Big deal. For most practical purposes, you already need a bunch of other
packages to do anything useful.

> > A minimal 'parse command line
> > to mount root and call real init' might be nice, but mostly as an
> > example, like the example watchdog code.
>
> See "partition detection in userspace".  How minimal is "minimal"?

Hello world, really. Given that kernels can't boot off IDE without a
bootloader, partition detection isn't going to get it closer to
self-booting.

> > I think a better way to go is to simply roll an initbootutils.tar.gz,
> > point to it in the kernel CHANGES, etc., start pulling stuff into it, and
> > people will catch on in no time.
>
> Possibly it could be gotten to work before being split off?  And then there's
> the question of upgrading your kernel without upgrading the initbootutils: if
> this never happens, what exactly is the benefit of it being in another
> tarball?  (I.E. how tight is the binding gonna be?  I certainly don't know
> yet...)

It ought not be any more tightly bound than regular libc. Isn't that the
point? If it still depends on non-generic services in the kernel, then we
haven't succeeded in pulling it all the way into userspace.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.."

Re: klibc development release

[Albert D. Cahalan]

Oliver Xymoron writes:
> On Sun, 11 Aug 2002, Rob Landley wrote:

>> How about partition detection?  When initramfs goes in that's one of the
>> things they're threatening to move to userspace.  Also lots of the hardware
>> detection and setup (ACPI, hotplug style PCI probing...)
...
> It ought not be any more tightly bound than regular libc. Isn't that the
> point? If it still depends on non-generic services in the kernel, then we
> haven't succeeded in pulling it all the way into userspace.

Klibc is a neat hack, but makes little sense. In the end we wind up
with _more_ code to maintain, not less. The boot process becomes
more complicated too. A microkernel by any other name still smells. :-)
Not all of us have 2 GHz boxes BTW.

Moving partition code out of the kernel is just begging for bugs
and limited functionality. The EVMS people have the right idea.
Does anyone else remember the user-space isapnp disaster? I do.
Users everywhere were screaming "my sound card won't work".

Re: klibc development release

[Rob Landley]

On Sunday 11 August 2002 02:20 pm, Albert D. Cahalan wrote:
> Oliver Xymoron writes:
> > On Sun, 11 Aug 2002, Rob Landley wrote:
> >> How about partition detection?  When initramfs goes in that's one of the
> >> things they're threatening to move to userspace.  Also lots of the
> >> hardware detection and setup (ACPI, hotplug style PCI probing...)
>
> ...
>
> > It ought not be any more tightly bound than regular libc.

"Regular" libc being glibc?  Which still supports 2.0 kernels by sacrificing 
chickens in the preprocessor, takes an insanely long time to compile (and 
with an invocation that's just a touch more complicated than "configure; 
make; make install"), and statically links "hello world" to 1.6 megabytes?

The system's libc is the glue between userspace and kernel space.  Every time 
the kernel comes up with new functionality (thread groups, 32 bit uids, 64 
bit file access...), userspace can't access it until libc gets up to speed.  
Pretending your libc isn't related to the kernel at all and can easily port 
to HP-UX and netbsd is just a recipe for bloat if you ask me.  Sometimes, 
being too generic is a BAD thing.

Personally, I question the klibc work a bit because it's reinventing the 
wheel with projects like uclibc already mostly there.  But Al pointed out the 
license issue with static linking against LGPL libraries, and in any case a 
stripped down minimalist libc as a reference implementation for others to 
build on is probably a good thing.  My interest in the libc area is a bit off 
to one side: I'm not fond of glibc and am looking to replace it in my own 
system, but it hasn't made it to the top of my to-do list yet.  (Dietlibc is 
straight GPL: it can't even be the dynamic replacement for glibc in a real 
world linux distribution.  HPA suggested I look at newlibc, which I've added 
to my to-do list).

> Isn't that the
> > point? If it still depends on non-generic services in the kernel, then we
> > haven't succeeded in pulling it all the way into userspace.
>
> Klibc is a neat hack, but makes little sense. In the end we wind up
> with _more_ code to maintain, not less. The boot process becomes
> more complicated too.

>From one perspective a little more complicated, but from another perspective 
a lot more understandable.  A lot more people understand userspace than 
understand kernel space, and a LOT more people are competent to hack 
userspace than kernel space.  (100 times as many people, at LEAST.  Even 
pinning down the order of magnitude depends on your definition of 
"competent", on both sides...)

> A microkernel by any other name still smells. :-)

A microkernel has multiple processes committing incest (ahem, "cooperating") 
spending all their time passing messages to each other, racing, and 
deadlocking on their own attempts at communication and synchronization.

Initramfs has a bunch of userspace processes initializing the system, but 
they mostly run one after the other in series and don't have to communicate 
with each other any more than the components of your average shell script.  
Basically, it takes the concept of init scripts and puts some of them before 
"init".  That isn't microkernel thinking, shell scripts are bog standard unix 
philosophy.

> Not all of us have 2 GHz boxes BTW.

My 133 mhz Dell laptop recently died, and got replaced by a 366 mhz toshiba.  
Woo, blazing speed.  My home desktop's still a pentium pro 180.  (I LIKE 
developing on slow machines: it makes you notice inefficient code and care 
about fixing it.  I have a 700 mhz athlon at work.  It's a file server.)

I'm kind of surprised people think an ACPI interpreter is going to depend on 
completely generic userspace services, but I suppose it takes all kinds.  (I 
can't say I'm an expert on ACPI, but from what I understand neither is 
anybody else, including Intel.)  And of course if you write devicefs so you 
can expose your device tree largely for power management purposes, and then 
retroactively call it a standard userspace service so you don't feel guilty 
when your ACPI interpreter interacts with it...  I'm all for it, it's a 
better API overall, but there's a bit of putting Descartes before the horse 
in there somewhere saying ACPI in userspace depends on generic services that 
are generic and exposed to userspace in part because you wanted to put ACPI 
in userspace long before the infrastructure to do so existed... :)

> Moving partition code out of the kernel is just begging for bugs
> and limited functionality. The EVMS people have the right idea.
> Does anyone else remember the user-space isapnp disaster? I do.
> Users everywhere were screaming "my sound card won't work".

The external pcmcia add-on package wasn't much fun either, and that required 
building against your kernel source directory...

Define "out of the kernel".  Looking at it one way, the nvidia driver is "out 
of kernel", even though it's a driver.  Looking at it another way, menuconfig 
is in-kernel.  It's in the tarball...

Doing stuff in process context can be very nice.  We have plenty of hybrid 
kernel/userspace things already: anything that shows up in ps with square 
brackets around it.  Initramfs puts more of the init-time stuff in process 
context, and the real REASON for doing this is again: more people grok 
userspace than grok an environment with spinlocks and preemption counts, 
where scheduling and swapping are not your friend, your stack is tiny, it's 
difficult to access the filesystem, you have to worry about what operations 
might conceivably block, reference counting and race conditions abound, you 
have to care about the difference between a virtual address and a physical 
one and then there's highmem and mapping to a PCI bus address...

There's simply a lot less to screw up in userspace.

That said, decoupling the reference implementation of the initramfs boot 
scripts from the kernel tarball is a seperate issue.  Thinking "we don't have 
to maintain that stuff, when it breaks somebody else will fix it" sounds fun, 
but is not always the world's most effective strategy.  If it stops being 
broken on a regular basis by new development, fine.  But I for one would 
prefer to wait for it to stabilize BEFORE trying to fork it off too far.

Of course it's not my call...

Rob

Re: klibc development release

[Erik Andersen]

On Sun Aug 11, 2002 at 11:31:13AM -0400, Rob Landley wrote:
> Personally, I question the klibc work a bit because it's reinventing the 
> wheel with projects like uclibc already mostly there.  But Al pointed out the 
> license issue with static linking against LGPL libraries, and in any case a 

Keep in mind that you _can_ staticly link even closed source
stuff against an LGPL library...  You just need to "Accompany the
work with a written offer, valid for at least three years" to
provide users with an object file that can be re-linked against
newer versions of the LGPL library.  Or simply provide the .o
file.  See the LGPL Section 6...

> to one side: I'm not fond of glibc and am looking to replace it in my own 
> system, but it hasn't made it to the top of my to-do list yet.  (Dietlibc is 
> straight GPL: it can't even be the dynamic replacement for glibc in a real 
> world linux distribution.  HPA suggested I look at newlibc, which I've added 
> to my to-do list).

As far as I know, uClibc is the only library that is able to
replace glibc for real world linux distributions...  And I've
looked long and hard (which was why I ended up making uClibc),

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: klibc development release

[H. Peter Anvin]

Erik Andersen wrote:
> 
>>to one side: I'm not fond of glibc and am looking to replace it in my own 
>>system, but it hasn't made it to the top of my to-do list yet.  (Dietlibc is 
>>straight GPL: it can't even be the dynamic replacement for glibc in a real 
>>world linux distribution.  HPA suggested I look at newlibc, which I've added 
>>to my to-do list).
> 
> As far as I know, uClibc is the only library that is able to
> replace glibc for real world linux distributions...  And I've
> looked long and hard (which was why I ended up making uClibc),
> 

This, of course, is not a goal for klibc at all.  I looked at uclibc,
dietlibc, and newlib before starting klibc.  klibc is meant to be *tiny*
first of all, and is very much not designed to be able to compile
arbitrary programs.

	-hpa

Re: klibc development release

[Jamie Lokier]

Rob Landley wrote:
> (Dietlibc is straight GPL: it can't even be the dynamic replacement
> for glibc in a real world linux distribution.  HPA suggested I look at
> newlibc, which I've added to my to-do list).

Since klibc is meant for compiling programs tightly bound to the kernel
in initramfs, such as partition scanning, NFS root mounting, module
loading etc., I wonder what the problem with even a GPLed klibc is?

Surely all the programs that its intended to be used with will be GPLed,
perhaps even part of the kernel source tree?

(Not that I mind at all.  I expect I will be using klibc's simplicity
and 3-clause BSD license in my employer's proprietary product soon enough..)

-- Jamie