klibc - another libc?

klibc - another libc?

[Michael Tokarev]

After several mentions of klibc recently, I want to ask a question.

I understand all the kernel-mode cleanups -- moving initialization
from kernel to user space is a very good thing.

But the question really is: why yet another libc?  We already have
dietlibc, uclibc, glibc, now klibc...  With modern kernel, initramfs
will very probably contain quite some programs linked with glibc
(modprobe/insmod, mdadm/lvm, etc; I highly suggest putting some
minimal text editor like nvi there too, for rescue purposes) --
so why not have an option to use whatever libc is available on
the host platform?

In the other words, kinit/ipconfig/nfsmount/etc stuff is ok,
no questions.  But the libc itself -- what for?

And another related question: why not dietlibc which is already
here, for quite long time?

Thanks.

/mjt

Re: klibc - another libc?

[H. Peter Anvin]

Followup to:  <44869397.4000907@tls.msk.ru>
By author:    Michael Tokarev <mjt@tls.msk.ru>
In newsgroup: linux.dev.kernel
>
> After several mentions of klibc recently, I want to ask a question.
> 
> I understand all the kernel-mode cleanups -- moving initialization
> from kernel to user space is a very good thing.
> 
> But the question really is: why yet another libc?  We already have
> dietlibc, uclibc, glibc, now klibc...  With modern kernel, initramfs
> will very probably contain quite some programs linked with glibc
> (modprobe/insmod, mdadm/lvm, etc; I highly suggest putting some
> minimal text editor like nvi there too, for rescue purposes) --
> so why not have an option to use whatever libc is available on
> the host platform?
> 

You have that option just fine; if you build your own initramfs you
can do whatever you want.

> In the other words, kinit/ipconfig/nfsmount/etc stuff is ok,
> no questions.  But the libc itself -- what for?

To be able to *require* it, which means it can't significantly bloat
the total size of the kernel image.  klibc binaries are *extremely*
small.  Static kinit is only a few tens of kilobytes, a lot of which
is zlib.

> And another related question: why not dietlibc which is already
> here, for quite long time?

- Bigger by an order of magnitude
- License issues
- Platform support
- Speed of portability (klibc is fully portable to a new platform in an afternoon)
- Additional issues which you can find if look through the archives of this list

	-hpa

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Wed, 7 Jun 2006, H. Peter Anvin wrote:

> To be able to *require* it, which means it can't significantly bloat
> the total size of the kernel image.  klibc binaries are *extremely*
> small.  Static kinit is only a few tens of kilobytes, a lot of which
> is zlib.

Every project starts small and has the annoying tendency to grow.
That still doesn't answer, why it has to be distributed with the kernel, 
just install the thing somewhere under /lib and Kbuild can link to it. The 
point is that it contains nothing kernel specific and doesn't has to be 
rebult with every new kernel.

bye, Roman

Re: klibc - another libc?

[H. Peter Anvin]

Followup to:  <Pine.LNX.4.64.0606080036250.17704@scrub.home>
By author:    Roman Zippel <zippel@linux-m68k.org>
In newsgroup: linux.dev.kernel
>
> Hi,
> 
> On Wed, 7 Jun 2006, H. Peter Anvin wrote:
> 
> > To be able to *require* it, which means it can't significantly bloat
> > the total size of the kernel image.  klibc binaries are *extremely*
> > small.  Static kinit is only a few tens of kilobytes, a lot of which
> > is zlib.
> 
> Every project starts small and has the annoying tendency to grow.
> That still doesn't answer, why it has to be distributed with the kernel, 
> just install the thing somewhere under /lib and Kbuild can link to it. The 
> point is that it contains nothing kernel specific and doesn't has to be 
> rebult with every new kernel.
> 

Actually, that isn't quite true.  One of the ways klibc is kept small
is by not guaranteeing a stable ABI... and not having compatibility
support for older kernels.  This is one of the kinds of luxuries that
bundling offers.

Does it make bundling mandatory?  Not really.  In fact, people have
been using klibc in its standalone form for years.  However, I believe
there would be a lot of resistace to have the kernel tarball have
outside dependencies on anything but the most basic build tools.

	-hpa

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Thu, 8 Jun 2006, H. Peter Anvin wrote:

> > That still doesn't answer, why it has to be distributed with the kernel, 
> > just install the thing somewhere under /lib and Kbuild can link to it. The 
> > point is that it contains nothing kernel specific and doesn't has to be 
> > rebult with every new kernel.
> > 
> 
> Actually, that isn't quite true.  One of the ways klibc is kept small
> is by not guaranteeing a stable ABI... and not having compatibility
> support for older kernels.  This is one of the kinds of luxuries that
> bundling offers.

It's indeed more of a luxury than a necessity.
How often does that really change? Bundling it with the kernel may 
actually encourage some developers to be less careful regarding 
compatibility. We already have enough problems with this as it is.

> Does it make bundling mandatory?  Not really.  In fact, people have
> been using klibc in its standalone form for years.  However, I believe
> there would be a lot of resistace to have the kernel tarball have
> outside dependencies on anything but the most basic build tools.

If you wouldn't remove all old init code at once it would still be 
possible to build a kernel this way. Why are you making it mandatory? Why 
don't you leave it optional for a while and only gradually remove the old 
code? This way distributions/users can experiment with it regarding their 
current initrd/boot setups.

Why shouldn't klibc be part of the basic build tools? I asked this already 
earlier: where do you draw the line regarding duplication? Are you going 
to duplicate every single tool, which might be needed to build the kernel 
only to reduce outside dependencies? IMO that's illusory for more complex 
setups anyway. Let's take booting from raid, in this case you need to 
install mdadm anyway, which could also provide an initramfs version. This 
way the setup tools can be generated from the same source, which reduces 
duplication and maintenance overhead.

Just to be clear here, I really appreciate the work you've done, but I'm 
not exactly comfortable with merging a huge patch, which completely 
changes the boot sequence at once, without any clear plan of what's coming 
next. It would be a lot less problematic if the transition would be more 
gradually, which IMO is very well possible. Usually it's a requirement to 
split large patches, why should klibc be special?

bye, Roman

Re: klibc - another libc?

[H. Peter Anvin]

Followup to:  <Pine.LNX.4.64.0606091409220.17704@scrub.home>
By author:    Roman Zippel <zippel@linux-m68k.org>
In newsgroup: linux.dev.kernel
> > 
> > Actually, that isn't quite true.  One of the ways klibc is kept small
> > is by not guaranteeing a stable ABI... and not having compatibility
> > support for older kernels.  This is one of the kinds of luxuries that
> > bundling offers.
> 
> It's indeed more of a luxury than a necessity.
> How often does that really change?

A lot more often than you'd think.

> If you wouldn't remove all old init code at once it would still be 
> possible to build a kernel this way. Why are you making it mandatory? Why 
> don't you leave it optional for a while and only gradually remove the old 
> code? This way distributions/users can experiment with it regarding their 
> current initrd/boot setups.

Linus vetoed that option years ago.  Anyway, the standalone version of
klibc has been available for almost three years, and has been used by
at least Debian for their initramfs code for a considerable amount of
time.  It's not like it's an untested piece of code.

> Why shouldn't klibc be part of the basic build tools? I asked this already 
> earlier: where do you draw the line regarding duplication?

Judgement call.  I'm not sure I have done the best possible job,
either, but you have to call it somewhere.

> Are you going to duplicate every single tool, which might be needed
> to build the kernel only to reduce outside dependencies? IMO that's
> illusory for more complex setups anyway.

Hyperbole.

FWIW, I've worked in environments where you pull a single source tree
and get the compilers as well as the code you build them with.  It is
*wonderful* for reproducibility, but I don't think that's going to
happen with respect to Linux, and noone is calling for that.

>  Let's take booting from raid, in this case you need to install
> mdadm anyway, which could also provide an initramfs version. This
> way the setup tools can be generated from the same source, which
> reduces duplication and maintenance overhead.

You don't need mdadm to boot from RAID.  kinit handles it just fine.
At the same time, I do intend to continue to maintain the standalone
klibc, which can be used to produce external binary trees.

> Just to be clear here, I really appreciate the work you've done, but I'm 
> not exactly comfortable with merging a huge patch, which completely 
> changes the boot sequence at once, without any clear plan of what's coming 
> next. It would be a lot less problematic if the transition would be more 
> gradually, which IMO is very well possible. Usually it's a requirement to 
> split large patches, why should klibc be special?

When I asked Andrew how he'd like me to pass him the code, he said he
was happy pulling my git tree.  This saved time for both of us, and
made it easier and quicker for me to integrate fixes.  You can do so
too, which will get you a full history of the development, in about
1,400 pieces.

	-hpa

Re: klibc - another libc?

[Michael Tokarev]

H. Peter Anvin wrote:
[]
>>  Let's take booting from raid, in this case you need to install
>> mdadm anyway, which could also provide an initramfs version. This
>> way the setup tools can be generated from the same source, which
>> reduces duplication and maintenance overhead.
> 
> You don't need mdadm to boot from RAID.  kinit handles it just fine.

You *do* need mdadm to boot from RAID.  Unless you rely on broken,
obsolete, "don't use" in-kernel raid autodetection code (which, in
this case, will be moved from kernel space into kinit).  There are
many reasons why raid autodetection in its current form is not a good
idea, all goes to simple "unreliable" definition - there where many
discussions about this already.

Well ok, mdadm/Assemble.c can be merged into kinit instead of current
stuff present there, and adopted somehow.  Until when, mdadm IS
necessary.

Ok, the next question may be 'and what about lvm?', or dm, or whatever
else..  Md autodetection code has been in kernel for a long time,
while lvm/dm/etc stuff wasn't.  So there IS a difference... ;)

But I see a reason for kinit *now*, in its current form - it's
compatibility.  Later on, maybe the whole stuff can be removed entirely,
to rely on external tools for booting.  Existing mkinitrd/mkinitramfs/
etc solutions works, they're being improved all the time, and they
don't use kinit.

Did I get it all right?  :)

Thanks.

/mjt

Re: klibc - another libc?

[H. Peter Anvin]

Michael Tokarev wrote:
> 
> You *do* need mdadm to boot from RAID.  Unless you rely on broken,
> obsolete, "don't use" in-kernel raid autodetection code (which, in
> this case, will be moved from kernel space into kinit).  There are
> many reasons why raid autodetection in its current form is not a good
> idea, all goes to simple "unreliable" definition - there where many
> discussions about this already.
> 
> Well ok, mdadm/Assemble.c can be merged into kinit instead of current
> stuff present there, and adopted somehow.  Until when, mdadm IS
> necessary.
> 
> Ok, the next question may be 'and what about lvm?', or dm, or whatever
> else..  Md autodetection code has been in kernel for a long time,
> while lvm/dm/etc stuff wasn't.  So there IS a difference... ;)
> 
> But I see a reason for kinit *now*, in its current form - it's
> compatibility.  Later on, maybe the whole stuff can be removed entirely,
> to rely on external tools for booting.  Existing mkinitrd/mkinitramfs/
> etc solutions works, they're being improved all the time, and they
> don't use kinit.
> 
> Did I get it all right?  :)
> 

Pretty much.  :)

However, this also is a good basis to point out that one of the big advantages with klibc 
is that you *can* share code with standalone userspace tools with relative ease.  For 
example, porting mdadm/Assemble.c to be a tool -- be it standalone or integrated into 
kinit -- is a pretty trivial project, whereas trying to integrate code like that into the 
kernel proper is a major task.

As you can also notice, a number of modules of kinit are also available as standalone 
programs (and, in fact, I have already gotten requests to break out additional modules in 
the same way.)  This means that that code is available to distributions to use in their 
initramfs solutions, without having to take the kinit package.  Finally, unlike the 
current code, kinit can be wrapped, as is, from a script.

	-hpa

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Fri, 9 Jun 2006, H. Peter Anvin wrote:

> > > Actually, that isn't quite true.  One of the ways klibc is kept small
> > > is by not guaranteeing a stable ABI... and not having compatibility
> > > support for older kernels.  This is one of the kinds of luxuries that
> > > bundling offers.
> > 
> > It's indeed more of a luxury than a necessity.
> > How often does that really change?
> 
> A lot more often than you'd think.

Often enough?

> > If you wouldn't remove all old init code at once it would still be 
> > possible to build a kernel this way. Why are you making it mandatory? Why 
> > don't you leave it optional for a while and only gradually remove the old 
> > code? This way distributions/users can experiment with it regarding their 
> > current initrd/boot setups.
> 
> Linus vetoed that option years ago.

Name dropping is of course always impressive - scares little kids and all.
Could you please provide more info, what exactly he vetoed?

>  Anyway, the standalone version of
> klibc has been available for almost three years, and has been used by
> at least Debian for their initramfs code for a considerable amount of
> time.  It's not like it's an untested piece of code.

I didn't say that, the point is if e.g. Debian is already distributing it, 
why do we need another copy?

> > Why shouldn't klibc be part of the basic build tools? I asked this already 
> > earlier: where do you draw the line regarding duplication?
> 
> Judgement call.  I'm not sure I have done the best possible job,
> either, but you have to call it somewhere.

Well for now you pretty much just moved code, that was in the kernel 
before. What I'm trying to find out is what is coming next. How does e.g. 
udev or modules fit into the picture?

> > Are you going to duplicate every single tool, which might be needed
> > to build the kernel only to reduce outside dependencies? IMO that's
> > illusory for more complex setups anyway.
> 
> Hyperbole.

Why?

> >  Let's take booting from raid, in this case you need to install
> > mdadm anyway, which could also provide an initramfs version. This
> > way the setup tools can be generated from the same source, which
> > reduces duplication and maintenance overhead.
> 
> You don't need mdadm to boot from RAID.  kinit handles it just fine.
> At the same time, I do intend to continue to maintain the standalone
> klibc, which can be used to produce external binary trees.

So why can't we use that external copy? mdadm is only example, there are 
more - please explain your plans. How will they be integrated?

> > Just to be clear here, I really appreciate the work you've done, but I'm 
> > not exactly comfortable with merging a huge patch, which completely 
> > changes the boot sequence at once, without any clear plan of what's coming 
> > next. It would be a lot less problematic if the transition would be more 
> > gradually, which IMO is very well possible. Usually it's a requirement to 
> > split large patches, why should klibc be special?
> 
> When I asked Andrew how he'd like me to pass him the code, he said he
> was happy pulling my git tree.  This saved time for both of us, and
> made it easier and quicker for me to integrate fixes.  You can do so
> too, which will get you a full history of the development, in about
> 1,400 pieces.

For -mm that's fine, but do you seriously expect it to get merged like 
that. Again, what makes klibc so special, that it doesn't have to follow 
standard rules?

Please be more specific, you are avoiding any clear answer. I'm really 
trying to understand, why you're doing it this way and what speaks against 
the alternatives and what are the next steps?

bye, Roman

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Fri, 9 Jun 2006, Michael Tokarev wrote:

> But I see a reason for kinit *now*, in its current form - it's
> compatibility.  Later on, maybe the whole stuff can be removed entirely,
> to rely on external tools for booting.

The compatibility code is already in the kernel, just don't call it if 
e.g. there's a /kinit in initramfs. We can add the hooks to the kernel to 
pull whatever you want into the initramfs and with time we can remove 
the old code.
Why create new (temporary?) compatibility code, if the current code is 
working just fine?

bye, Roman

Re: klibc - another libc?

[Sam Ravnborg]

> Well for now you pretty much just moved code, that was in the kernel 
> before. What I'm trying to find out is what is coming next. How does e.g. 
> udev or modules fit into the picture?
udev and module-init-tools fits nicely with the kernel. I never have
understood this 'keep-everyhig-separate' mantra. Just see how many
people had troubles with missing module-init-tools or people having
troubles with non-backward compatible udev.

> For -mm that's fine, but do you seriously expect it to get merged like 
> that. Again, what makes klibc so special, that it doesn't have to follow 
> standard rules?
So part of what you ask for is a set of incremental patches that shows
all the kernal modifications?
That should be doable with some effort but unfortunately I'm not up to
the task.

	Sam

Re: klibc - another libc?

[Michael Tokarev]

Roman Zippel wrote:
> Hi,
> 
> On Fri, 9 Jun 2006, Michael Tokarev wrote:
> 
>> But I see a reason for kinit *now*, in its current form - it's
>> compatibility.  Later on, maybe the whole stuff can be removed entirely,
>> to rely on external tools for booting.
> 
> The compatibility code is already in the kernel, just don't call it if 

Isn't kinit/klibc intended to *replace* that whole code in the kernel?

> e.g. there's a /kinit in initramfs. We can add the hooks to the kernel to 
> pull whatever you want into the initramfs and with time we can remove 
> the old code.
> Why create new (temporary?) compatibility code, if the current code is 
> working just fine?

That was partly my question (or something I had in mind but didn't ask).

Well.  It's interesting.

Embedded folks are using uclibc or dietlibc - i see no reason for another
"more small" libc for booting, regular tools (linked against uc or diet)
can be used just fine.

Big systems will usually require some "advanced" stuff like that same
mdadm or lvm - and again, I don't see why regular glibc can't be used
(wich has an advandage - no need to have two versions of the toolset,
linked with klibc and glibc).

The only real reason I see in klibc is to be able to remove all that
'kinit' code from kernel but still be compatible - so that I will
be able to download linux-2.7, make oldconfig all install, and it
Will Just Work (in case I had no initrd/initramfs) - new userspace
kinit will take care of everything removed from kernel code.

In the other words, i see the whole klibc thing as a temporary
compatibility solution.

Did I miss something?

And yes, it's a PITA to have two sets of all the system utilities -
I'd not go this route.  Also, since klibc will have compatibility
problems with kernel API changes, such a 'parallel toolset' needs
to be recompiled on a regular basis, which means not two toolsets,
but many of them (one 'main', and several 'small for boot', for
different kernel versions, linked against different klibc-XXX.so).

The only problem I see with using glibc in initramfs is -- systems
with small amount of memory, where full-blown glibc with all the
required tools will just not fit into ramfs/tmpfs.  Not "embedded"
systems (like boxed routers for example - they're already using
something much more compact than glibc).  Are there such systems
out there still?

/mjt

Re: klibc - another libc?

[Sam Ravnborg]

On Sat, Jun 10, 2006 at 08:24:06PM +0400, Michael Tokarev wrote:
 
> Embedded folks are using uclibc or dietlibc - i see no reason for another
> "more small" libc for booting, regular tools (linked against uc or diet)
> can be used just fine.
Then they can just use uclibc or dietlibc. There is nothing that
prevents you from providing everything fr the initramfs.
klibc is the default choice so the kernel can provide a functional set
of programs. Without klibc it would be almost impossible to factor out
all the code that ought to run in userspace but are part of the ekrnel
because there were no other way to do it.

And klibc being temporary is some odd argument. Why should it be
temporary. Are there other alternatives when bundling a 'libc' with the
kernel?

	Sam

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Sat, 10 Jun 2006, Sam Ravnborg wrote:

> > Well for now you pretty much just moved code, that was in the kernel 
> > before. What I'm trying to find out is what is coming next. How does e.g. 
> > udev or modules fit into the picture?
> udev and module-init-tools fits nicely with the kernel. I never have
> understood this 'keep-everyhig-separate' mantra. Just see how many
> people had troubles with missing module-init-tools or people having
> troubles with non-backward compatible udev.

It just means that we suck at producing stable kernel interfaces.
Moving everything into the kernel doesn't magically solve problem, it only 
shifts the problem.
You still had to define the boundaries - how does it interface with the 
rest of the system. This is the part I'm currently trying to figure out 
regarding klibc.

> > For -mm that's fine, but do you seriously expect it to get merged like 
> > that. Again, what makes klibc so special, that it doesn't have to follow 
> > standard rules?
> So part of what you ask for is a set of incremental patches that shows
> all the kernal modifications?

Well, it's common practice. If we start making exceptions, everyone wants 
one, so it IMO would need a very, very good reason to merge this as is.

bye, Roman

Re: klibc - another libc?

[Roman Zippel]

Hi,

On Sat, 10 Jun 2006, Michael Tokarev wrote:

> >> But I see a reason for kinit *now*, in its current form - it's
> >> compatibility.  Later on, maybe the whole stuff can be removed entirely,
> >> to rely on external tools for booting.
> > 
> > The compatibility code is already in the kernel, just don't call it if 
> 
> Isn't kinit/klibc intended to *replace* that whole code in the kernel?

Well, that's the basic idea, but slowly I'm wondering whether anyone has 
really figured out how to do this.
Being able to produce small binaries is part of it, but it somehow also 
has to work together with the rest of system, e.g. how can be configured 
(by the user, distribution, ...) or how can it be extended.
We need to know where we are going, otherwise it's hard to know whether 
we're on the right path.

bye, Roman

Re: klibc - another libc?

[Paul Dickson]

On Sat, 10 Jun 2006 03:15:59 +0200 (CEST), Roman Zippel wrote:

> On Fri, 9 Jun 2006, H. Peter Anvin wrote:
> 
> > > If you wouldn't remove all old init code at once it would still be 
> > > possible to build a kernel this way. Why are you making it mandatory? Why 
> > > don't you leave it optional for a while and only gradually remove the old 
> > > code? This way distributions/users can experiment with it regarding their 
> > > current initrd/boot setups.
> > 
> > Linus vetoed that option years ago.
> 
> Name dropping is of course always impressive - scares little kids and all.
> Could you please provide more info, what exactly he vetoed?

His rule was no code was to be added to the kernel code unless it already
had a user.  So there would be no adding code hoping a user would appear.

If I'm wrong, I'll likely be corrected... :-)

	-Paul