iproute2 and kernel headers

iproute2 and kernel headers

[Stephen Hemminger]

I am willing to put some headers (not all) in with the user level
code, provided they are copies since they I can easily update. I don't want
to get into keeping an edited set of headers in sync.

What headers really seem to change a lot? The obvious ones are:

	linux/pkt_sched.h
	linux/tcp_diag.h
	linux/xfrm.h

Re: iproute2 and kernel headers

[David S. Miller]

On Mon, 2 Aug 2004 15:38:05 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> I am willing to put some headers (not all) in with the user level
> code, provided they are copies since they I can easily update. I don't want
> to get into keeping an edited set of headers in sync.
> 
> What headers really seem to change a lot? The obvious ones are:
> 
> 	linux/pkt_sched.h
> 	linux/tcp_diag.h
> 	linux/xfrm.h

I would add linux/rtnetlink.h

Re: iproute2 and kernel headers

[Tomasz Torcz]

On Mon, Aug 02, 2004 at 03:38:05PM -0700, Stephen Hemminger wrote:
> I am willing to put some headers (not all) in with the user level
> code, provided they are copies since they I can easily update. I don't want
> to get into keeping an edited set of headers in sync.

 Aren't linux-libc-headers (*) sufficient?

* - http://ep09.pld-linux.org/~mmazur/linux-libc-headers/

-- 
Tomasz Torcz                        To co nierealne - tutaj jest normalne.
zdzichu@irc.-nie.spam-.pl          Ziomale na życie mają tu patenty specjalne.

Re: iproute2 and kernel headers

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20040802160445.5ef3b251.davem@redhat.com> (at Mon, 2 Aug 2004 16:04:45 -0700), "David S. Miller" <davem@redhat.com> says:

> > What headers really seem to change a lot? The obvious ones are:
> > 
> > 	linux/pkt_sched.h
> > 	linux/tcp_diag.h
> > 	linux/xfrm.h
> 
> I would add linux/rtnetlink.h

Then, linux/snmp.h.

--yoshfuji

Re: iproute2 and kernel headers

[Stephen Hemminger]

On Tue, 3 Aug 2004 02:14:59 +0200
Tomasz Torcz <zdzichu@irc.pl> wrote:

> On Mon, Aug 02, 2004 at 03:38:05PM -0700, Stephen Hemminger wrote:
> > I am willing to put some headers (not all) in with the user level
> > code, provided they are copies since they I can easily update. I don't want
> > to get into keeping an edited set of headers in sync.
> 
>  Aren't linux-libc-headers (*) sufficient?
> 
> * - http://ep09.pld-linux.org/~mmazur/linux-libc-headers/

The theory of that is good, but in practice it would make the problem worse.
What iproute2 wants is to have the same kernel data structures as the latest kernel.
It is awkward enough making sure to get them from the correct kernel sources, but
doing it from a different package would make updating and keeping everything current worse.

Re: iproute2 and kernel headers

[Mariusz Mazur]

On wtorek 03 sierpień 2004 17:31, Stephen Hemminger wrote:
> > > I am willing to put some headers (not all) in with the user level
> > > code, provided they are copies since they I can easily update. I don't
> > > want to get into keeping an edited set of headers in sync.
> >
> >  Aren't linux-libc-headers (*) sufficient?
> >
> > * - http://ep09.pld-linux.org/~mmazur/linux-libc-headers/
>
> The theory of that is good, but in practice it would make the problem
> worse. What iproute2 wants is to have the same kernel data structures as
> the latest kernel. It is awkward enough making sure to get them from the
> correct kernel sources, but doing it from a different package would make
> updating and keeping everything current worse.

This message was forwarded to me (I'm the llh maintainer). Llh is being 
updated by generating a diff between incremental kernel releases, fixing that 
diff and applying it to the last version of llh. I do have scripts to check 
wether such an update doesn't break any headers. There is a slight 
possibility that such an update might omit some crucial changes, but (at 
least when it comes to network headers) co-developers of my distribution 
(PLD) will most likely detect that allmost instantly and notify me what's 
wrong (in such a case I will release a fixed version of llh asap).

Since removing all those glibc/linux-headers workarounds from 
iproute2/iptables was quite a pain for me I would really like to see building 
against llh at least as an option available from 'configure'.
(In case you're wondering - yes, we've been building all the linux network 
userland stuff against llh for about 8 months now without any problems... 
quite the opposite actually)


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again

Re: iproute2 and kernel headers

[David S. Miller]

On Tue, 03 Aug 2004 00:55:42 -0700 (PDT)
YOSHIFUJI Hideaki / ^[$B5HF#1QL@^[(B <yoshfuji@linux-ipv6.org> wrote:

> In article <20040802160445.5ef3b251.davem@redhat.com> (at Mon, 2 Aug 2004 16:04:45 -0700), "David S. Miller" <davem@redhat.com> says:
> 
> > > What headers really seem to change a lot? The obvious ones are:
> > > 
> > > 	linux/pkt_sched.h
> > > 	linux/tcp_diag.h
> > > 	linux/xfrm.h
> > 
> > I would add linux/rtnetlink.h
> 
> Then, linux/snmp.h.

Agreed.

Re: iproute2 and kernel headers

[Jean Tourrilhes]

Stephen Hemminger wrote :
> 
> On Tue, 3 Aug 2004 02:14:59 +0200 Tomasz Torcz <zdzichu@irc.pl> wrote:
> 
> > On Mon, Aug 02, 2004 at 03:38:05PM -0700, Stephen Hemminger wrote:
> > > I am willing to put some headers (not all) in with the user level
> > > code, provided they are copies since they I can easily update. I don't want
> > > to get into keeping an edited set of headers in sync.
> > 
> >  Aren't linux-libc-headers (*) sufficient?
> > 
> > * - http://ep09.pld-linux.org/~mmazur/linux-libc-headers/
> 
> The theory of that is good, but in practice it would make the problem
> worse.  What iproute2 wants is to have the same kernel data structures
> as the latest kernel.  It is awkward enough making sure to get them
> from the correct kernel sources, but doing it from a different package
> would make updating and keeping everything current worse.

	Hi Stephen,

	Sorry to bring bad news, but I wanted to share my experience
on the subject with you. This e-mail might not give you solution, but
I hope it will at least entertain you ;-)


	I have the same problem with Wireless Extensions. My thinking
was similar to yours a while back, and since then I've tried various
solutions. My personal opinion on the subject is that you have just
opened a big can of worms, and you will quickly realise that it's
worse than you initally though...

	Back when I started with Wireless Extensions, the kernel
headers in /usr/include/linux where a symbolic link to the actual
kernel headers. I was just building Wireless Tools against
/usr/include/linux/wireless.h, and magically both the tools and the
kernel were using the same defintions of various data structures.

	Then, people started to upgrade their kernel and forgot to
recompile the tools. Personally, the first thing I do when I install a
Debian stable to to wipe out the obsolete kernel and install the
latest. The end result was that the tools were crashing in all kind of
mysterious ways, because the data structs were not the same size in
the tools and the kernel.
	I learned pretty quickly that version number on the tools were
not much help. If kernel version X recommend tools version A, but
tools version A were compiled for kernel version Y, the tools will
still malfunction. And the user won't understand because he has the
"right" version.
	I tried to educate users, with notices in README and on my web
page, but after answering the same question for hundreds of time, I
felt I was wasting my time. I don't think it's possible to expect
standard users to always recompile those system tools when they
upgrade the kernel.

	This is when I introduced version number in the API. This
allows the tools to verify that they use the same version of the API
as the kernel, and complain if they are not with an explicit error
message.
	Things were somewhat better, at least the errors were no
longer mysterious and a significant number of users figured out how to
fix it by themselves.
	However, many users got very confused by the two version
numbers and their relationship.

	Around this time, /usr/include/linux was no longer a symlink
and became independant of the kernel version (what we have
today). That was another additional annoyance.
	There are some ways to get around this. One way is to poke
directly in /usr/src/linux for the header. Another is to have local
copies of all the versions of the header, and poke the current running
kernel for the API version number to select the proper version of the
header. Or you can let the user decide which version to use at
configure time. I implemented all three for wireless tools.
	Your proposal would fix this precise issue, but in the grand
scheme of things, I believe this is only one annoyance, not the main
issue. You still have data structure mismatches if the kernel and the
tools are not compiled with the same version of the kernel headers.

	By the way, one of the consequence of versioning the API is
that I tend to do most API changes in batches. The idea is that I want
to minimise the number of API versions, because I have to test the
tools and drivers with each of them, and I have finite time.
	The kernel people hate me, because I submit changes as big
batches, which is contrary to the "small patch" philosophy. The
driver/tools people hate me because the changes they depend upon are
queued up until next batch *and* they have to test for multiple
versions of the API and #define all over the place in their code. You
can't win...

	Soon after that, the distribution people started to hate
me. Many of their users were seeing my message about API mismatche and
rep[orting it as a bug. Distro only offer one prepackaged version of
the tools ; Some distro offer multiple kernel options (Debian), or
offer prepackaged kernel updates (Red-Hat). Of course, the tools would
match only one version of the kernel, and not the other ones. Distro
went as far as just disable my error message, because it was confusing
their users too much, and live with the broken functionality.
	Let's also not forget people like me having multiple kernels
on the same system (like one 2.4.X and one 2.6.X kernels in my
case). You don't want to recompile your tools each time you boot the
other kernel.

	I tried to have a system where I would compile the tools with
different versions of the API (i.e. create multiple binaries), and
have a small dispatcher that would call the right version of the tools
depending on the running kernel. That was working, but just ugly and
fragile.
	The current solution is that I implemented a layer that
translate those data structures from one version to another. I read
the version of the API and translate the structure to whatever the
tool is expecting. Basically, a single binary dynamically support
multiple versions of the API and multiple kernels.
	This works for me because the number of API versions and data
structures is limited. I'm not sure if it would work for everybody...

	Now, that's my story. Other people have their own stories, and
for example I would ask the netfilter people how they deal with this
problem.

	Now, my sick brain would like to suggest a few other crazy
solutions totally out of the box :

	1) Get it right the first time, so no change is ever
needed. If things are standardised (POSIX, IEEE), this can work, if
you are called Linus, most likely, otherwise, good luck...

	2) Migrate all those kernel APIs to XML. XML is the future,
you can validate the output, and you can add/remove nodes/attributes
between versions without the full thing falling apart. And we already
have a web server in the kernel.

	3) Every time you need to change the API, migrate it to
another delivery mechanism and create a totally different set of
tools. If you were using ioctl, migrate to netlink. If you were using
netlink, migrate to a pseudo filesystem. Obviously the holy grail is
to make it a system call. Also, make sure to kill the old APIs and old
tools, otherwise some other folks my continue to maintain it.

	4) Distribute all those system utilities as part of the
kernel, compile them as part of the kernel, and install them in a
kernel specific directory. Basically, treat system utilities exactly
the same way as kernel modules.
	That actually would go a long way toward solving the issue of
distribution of system utils (where is module-init-tools ?) and push
more people to implement their pet project in user space rather than
in the kernel.

	I wish you a lot of luck exploring those issues, and if one of
those long nights you have a Eureka moment, please share with us ;-)
	Have fun...

	Jean

Re: iproute2 and kernel headers

[Stephen Hemminger]

On Wed, 4 Aug 2004 17:50:19 -0700
Jean Tourrilhes <jt@bougret.hpl.hp.com> wrote:

> Stephen Hemminger wrote :
> > 
> > On Tue, 3 Aug 2004 02:14:59 +0200 Tomasz Torcz <zdzichu@irc.pl> wrote:
> > 
> > > On Mon, Aug 02, 2004 at 03:38:05PM -0700, Stephen Hemminger wrote:
> > > > I am willing to put some headers (not all) in with the user level
> > > > code, provided they are copies since they I can easily update. I don't want
> > > > to get into keeping an edited set of headers in sync.
> > > 
> > >  Aren't linux-libc-headers (*) sufficient?
> > > 
> > > * - http://ep09.pld-linux.org/~mmazur/linux-libc-headers/
> > 
> > The theory of that is good, but in practice it would make the problem
> > worse.  What iproute2 wants is to have the same kernel data structures
> > as the latest kernel.  It is awkward enough making sure to get them
> > from the correct kernel sources, but doing it from a different package
> > would make updating and keeping everything current worse.
> 
> 	Hi Stephen,
> 
> 	Sorry to bring bad news, but I wanted to share my experience
> on the subject with you. This e-mail might not give you solution, but
> I hope it will at least entertain you ;-)
> 
> 
> 	I have the same problem with Wireless Extensions. My thinking
> was similar to yours a while back, and since then I've tried various
> solutions. My personal opinion on the subject is that you have just
> opened a big can of worms, and you will quickly realise that it's
> worse than you initally though...
> 
> 	Back when I started with Wireless Extensions, the kernel
> headers in /usr/include/linux where a symbolic link to the actual
> kernel headers. I was just building Wireless Tools against
> /usr/include/linux/wireless.h, and magically both the tools and the
> kernel were using the same defintions of various data structures.
> 
> 	Then, people started to upgrade their kernel and forgot to
> recompile the tools. Personally, the first thing I do when I install a
> Debian stable to to wipe out the obsolete kernel and install the
> latest. The end result was that the tools were crashing in all kind of
> mysterious ways, because the data structs were not the same size in
> the tools and the kernel.
> 	I learned pretty quickly that version number on the tools were
> not much help. If kernel version X recommend tools version A, but
> tools version A were compiled for kernel version Y, the tools will
> still malfunction. And the user won't understand because he has the
> "right" version.
> 	I tried to educate users, with notices in README and on my web
> page, but after answering the same question for hundreds of time, I
> felt I was wasting my time. I don't think it's possible to expect
> standard users to always recompile those system tools when they
> upgrade the kernel.
> 
> 	This is when I introduced version number in the API. This
> allows the tools to verify that they use the same version of the API
> as the kernel, and complain if they are not with an explicit error
> message.
> 	Things were somewhat better, at least the errors were no
> longer mysterious and a significant number of users figured out how to
> fix it by themselves.
> 	However, many users got very confused by the two version
> numbers and their relationship.
> 
> 	Around this time, /usr/include/linux was no longer a symlink
> and became independant of the kernel version (what we have
> today). That was another additional annoyance.
> 	There are some ways to get around this. One way is to poke
> directly in /usr/src/linux for the header. Another is to have local
> copies of all the versions of the header, and poke the current running
> kernel for the API version number to select the proper version of the
> header. Or you can let the user decide which version to use at
> configure time. I implemented all three for wireless tools.
> 	Your proposal would fix this precise issue, but in the grand
> scheme of things, I believe this is only one annoyance, not the main
> issue. You still have data structure mismatches if the kernel and the
> tools are not compiled with the same version of the kernel headers.
> 
> 	By the way, one of the consequence of versioning the API is
> that I tend to do most API changes in batches. The idea is that I want
> to minimise the number of API versions, because I have to test the
> tools and drivers with each of them, and I have finite time.
> 	The kernel people hate me, because I submit changes as big
> batches, which is contrary to the "small patch" philosophy. The
> driver/tools people hate me because the changes they depend upon are
> queued up until next batch *and* they have to test for multiple
> versions of the API and #define all over the place in their code. You
> can't win...
> 
> 	Soon after that, the distribution people started to hate
> me. Many of their users were seeing my message about API mismatche and
> rep[orting it as a bug. Distro only offer one prepackaged version of
> the tools ; Some distro offer multiple kernel options (Debian), or
> offer prepackaged kernel updates (Red-Hat). Of course, the tools would
> match only one version of the kernel, and not the other ones. Distro
> went as far as just disable my error message, because it was confusing
> their users too much, and live with the broken functionality.
> 	Let's also not forget people like me having multiple kernels
> on the same system (like one 2.4.X and one 2.6.X kernels in my
> case). You don't want to recompile your tools each time you boot the
> other kernel.
> 
> 	I tried to have a system where I would compile the tools with
> different versions of the API (i.e. create multiple binaries), and
> have a small dispatcher that would call the right version of the tools
> depending on the running kernel. That was working, but just ugly and
> fragile.
> 	The current solution is that I implemented a layer that
> translate those data structures from one version to another. I read
> the version of the API and translate the structure to whatever the
> tool is expecting. Basically, a single binary dynamically support
> multiple versions of the API and multiple kernels.
> 	This works for me because the number of API versions and data
> structures is limited. I'm not sure if it would work for everybody...
> 
> 	Now, that's my story. Other people have their own stories, and
> for example I would ask the netfilter people how they deal with this
> problem.
> 
> 	Now, my sick brain would like to suggest a few other crazy
> solutions totally out of the box :
> 
> 	1) Get it right the first time, so no change is ever
> needed. If things are standardised (POSIX, IEEE), this can work, if
> you are called Linus, most likely, otherwise, good luck...

The management API's are what seemed to get changed the most.
The only relevant standard's seem to be SNMP, but it doesn't match
the kernel API needs.

> 	2) Migrate all those kernel APIs to XML. XML is the future,
> you can validate the output, and you can add/remove nodes/attributes
> between versions without the full thing falling apart. And we already
> have a web server in the kernel.

No, don't reinvent SOAP for kernel API's.

> 	3) Every time you need to change the API, migrate it to
> another delivery mechanism and create a totally different set of
> tools. If you were using ioctl, migrate to netlink. If you were using
> netlink, migrate to a pseudo filesystem. Obviously the holy grail is
> to make it a system call. Also, make sure to kill the old APIs and old
> tools, otherwise some other folks my continue to maintain it.

Doesn't make sense unless the new interface is better.
Just use different ioctl's or message types.

> 	4) Distribute all those system utilities as part of the
> kernel, compile them as part of the kernel, and install them in a
> kernel specific directory. Basically, treat system utilities exactly
> the same way as kernel modules.
> 	That actually would go a long way toward solving the issue of
> distribution of system utils (where is module-init-tools ?) and push
> more people to implement their pet project in user space rather than
> in the kernel.

Too old school, unix.  It might have worked in the past, and we might
end up back there, but it ain't going to happen now.


> 	I wish you a lot of luck exploring those issues, and if one of
> those long nights you have a Eureka moment, please share with us ;-)
> 	Have fun...

You ignored the advantage of using simple string interfaces (a.l.a Plan 9)
or simple name:value pairs. Also, it turns out that some simple things like
increasing the size of the structure or adding a new rtnetlink message type
are not too hard to deal with.

Re: iproute2 and kernel headers

[Jean Tourrilhes]

On Fri, Aug 06, 2004 at 09:39:20AM -0700, Stephen Hemminger wrote:
> On Wed, 4 Aug 2004 17:50:19 -0700
> Jean Tourrilhes <jt@bougret.hpl.hp.com> wrote:
> > 
> > 	Now, my sick brain would like to suggest a few other crazy
> > solutions totally out of the box :
> > 
> > 	1) Get it right the first time, so no change is ever
> > needed. If things are standardised (POSIX, IEEE), this can work, if
> > you are called Linus, most likely, otherwise, good luck...
> 
> The management API's are what seemed to get changed the most.
> The only relevant standard's seem to be SNMP, but it doesn't match
> the kernel API needs.

	And SNMP standardisation usually is late to the party.

> > 	2) Migrate all those kernel APIs to XML. XML is the future,
> > you can validate the output, and you can add/remove nodes/attributes
> > between versions without the full thing falling apart. And we already
> > have a web server in the kernel.
> 
> No, don't reinvent SOAP for kernel API's.

	I feel honored that you are taking my braindamage so
seriously. Maybe I should have suggested to have a Lisp command line
to configure the kernel (angle brackets or parenthesis, all the same
to me).
	The beauty of XML is that you can use any gramar you want and
still be compliant. Obviously, we have to define our own gramar, named
LKXML :
	<Linus>
		<Proc>i386</Proc>
		<JeffG>
			<eth0 full_duplex="on"/>
		</JeffG>
		<DaveM>
			<TCPVegas Alpha="2" Beta="6"/>
		</DaveM>
		<RMK>
			<arch>StrongARM</arch>
		</RMK>
		<StephenH>
			<Bug Text="Incompatible statements" Severity="42">
				<Proc>i386</Proc>
				<arch>StrongARM</arch>
			</Bug>
		</StephenH>
	</Linus>

> > 	3) Every time you need to change the API, migrate it to
> > another delivery mechanism and create a totally different set of
> > tools. If you were using ioctl, migrate to netlink. If you were using
> > netlink, migrate to a pseudo filesystem. Obviously the holy grail is
> > to make it a system call. Also, make sure to kill the old APIs and old
> > tools, otherwise some other folks my continue to maintain it.
> 
> Doesn't make sense unless the new interface is better.
> Just use different ioctl's or message types.

	Hum... Every time I will change the data struct (add a field,
change a field type), I will move to a new ioctl number. Maybe we want
to migrate ioctl numbers from 32 bits to 64 bits to avoid quickly
exhausting the ioctl space.

> > 	4) Distribute all those system utilities as part of the
> > kernel, compile them as part of the kernel, and install them in a
> > kernel specific directory. Basically, treat system utilities exactly
> > the same way as kernel modules.
> > 	That actually would go a long way toward solving the issue of
> > distribution of system utils (where is module-init-tools ?) and push
> > more people to implement their pet project in user space rather than
> > in the kernel.
> 
> Too old school, unix.  It might have worked in the past, and we might
> end up back there, but it ain't going to happen now.

	I pretty much test every kernel release, and every month I
need to go in /lib/modules and remove obsolete modules directories to
free some disk space. The idea of having a new version of
ifconfig/iwconfig/iproute/cardmgr/... on my hard disk every time I
upgrade my kernel doesn't appeal to me.
	Note : this is why I kill the wireless redirector stuff.

> > 	I wish you a lot of luck exploring those issues, and if one of
> > those long nights you have a Eureka moment, please share with us ;-)
> > 	Have fun...
> 
> You ignored the advantage of using simple string interfaces (a.l.a Plan 9)
> or simple name:value pairs.

	Yep, you can do everything with strings, even introduces
parsing bugs in the kernel (been there...). It's also fun dealing with
fields that contain spaces or fancy character in them, fortunately it
only happen when the API is specified by non-Unix standards. And there
is so many formating of simple name:value pairs (with ':', '=' or ' '
as separator). By the way, as far as I am concerned, XML is just a
fancy name:value pairs format with angle brackets (see above).
	Also, it depends if you care about kernel bloat. Maybe we
should make kernel memory swapable.

	Note : I have some kernel APIs that are ASCII based. It's
worth it because you don't need user tools at all. But unfortunately
it's not always applicable, otherwise stuff like RtNetlink that don't
require backward compatibility would have been designed ASCII based.

> Also, it turns out that some simple things like
> increasing the size of the structure or adding a new rtnetlink message type
> are not too hard to deal with.

	Actually, reading a struct that increased in size will make
your read buffer overflow if you are not careful. Note that this is
the solution I adopted in the end, but as I say, you need to be
careful.
	Note that with respect to your *original* problem (yeah, we
are so slightly off-topic), if this is the case you only need to
include the very latest version of the header, and don't really care
what is the actual header in the kernel.

	Have fun...

	Jean

Re: iproute2 and kernel headers

[jamal]

On Fri, 2004-08-06 at 12:39, Stephen Hemminger wrote: 
> On Wed, 4 Aug 2004 17:50:19 -0700
> Jean Tourrilhes <jt@bougret.hpl.hp.com> wrote:

[..]

> > 	This is when I introduced version number in the API. This
> > allows the tools to verify that they use the same version of the API
> > as the kernel, and complain if they are not with an explicit error
> > message.

Yes, this is useful. Unfortunately this gives you control only on your
API. If for example someone fscks something you are using, such as the
netlink header you are screwed. So for consistency versioning needs to
exist everywhere.


> > 	By the way, one of the consequence of versioning the API is
> > that I tend to do most API changes in batches. The idea is that I want
> > to minimise the number of API versions, because I have to test the
> > tools and drivers with each of them, and I have finite time.

There should really be no reason you have to change versions. It should
be the last resort. You can use tricks like data structure augmentation
and and new TLV types to go for a long time and still be backward (as
well as forward) compatible. When you are no longer capable of doing
these tricks, then it would make sense upping the version.
There also should be rules for evolution reasons against having data
structures which cross kernel/userspace from having things like 
lookatme[0] elements.



 
> > 	1) Get it right the first time, so no change is ever
> > needed. If things are standardised (POSIX, IEEE), this can work, if
> > you are called Linus, most likely, otherwise, good luck...
> 
> The management API's are what seemed to get changed the most.
> The only relevant standard's seem to be SNMP, but it doesn't match
> the kernel API needs.

I think we could establish certain rules that will make things live for
a long period of time. Some which have been pointed to here already are:

a) use TLVs
b) When you really wanna transport new things, create new Types leaving
old ones intact so older versions of tools dont break.
c) Use data structure augmentation i.e add things at the end of a data
structure. In combination with sizeof operations old tools should work
while new ones can make use of new features.
d) avoid like the plague things like lookatme[0] elements within a
structure that gets transfered between kernel/userspace

> > 	2) Migrate all those kernel APIs to XML. XML is the future,
> > you can validate the output, and you can add/remove nodes/attributes
> > between versions without the full thing falling apart. And we already
> > have a web server in the kernel.
> 
> No, don't reinvent SOAP for kernel API's.

We already have value=attribute in TLVs approach (which could be
nested). 
Your userspace app should be able to use XML and use some DTD to
translate into binary TLV repreasentations.

> > 	3) Every time you need to change the API, migrate it to
> > another delivery mechanism and create a totally different set of
> > tools. If you were using ioctl, migrate to netlink. If you were using
> > netlink, migrate to a pseudo filesystem. Obviously the holy grail is
> > to make it a system call. Also, make sure to kill the old APIs and old
> > tools, otherwise some other folks my continue to maintain it.
> 
> Doesn't make sense unless the new interface is better.
> Just use different ioctl's or message types.

Refer to the rules above.

> > 	4) Distribute all those system utilities as part of the
> > kernel, compile them as part of the kernel, and install them in a
> > kernel specific directory. Basically, treat system utilities exactly
> > the same way as kernel modules.
> > 	That actually would go a long way toward solving the issue of
> > distribution of system utils (where is module-init-tools ?) and push
> > more people to implement their pet project in user space rather than
> > in the kernel.
> 
> Too old school, unix.  It might have worked in the past, and we might
> end up back there, but it ain't going to happen now.

My thought (i have proposed this a few times, probably not eloquently)
is to build discovery. User space should be able to discover at runtime
what the kernel can do. X for example does this in a stupid way: it
attempts featuress and on failure it takes them off its list
.
My approach is to explicitly tell the kernel or ask the kernel
"tell me what you can do"  - this should bring up things like
capabilities and revisions of different things in the kernel.
User space should then adapt accordingly.

> > 	I wish you a lot of luck exploring those issues, and if one of
> > those long nights you have a Eureka moment, please share with us ;-)
> > 	Have fun...
> 
> You ignored the advantage of using simple string interfaces (a.l.a Plan 9)
> or simple name:value pairs. Also, it turns out that some simple things like
> increasing the size of the structure or adding a new rtnetlink message type
> are not too hard to deal with.

Perhaps we need to start accumulating the rules. Ive listed a few - some of 
which you list above.

cheers,
jamal

Re: iproute2 and kernel headers

[Mariusz Mazur]

On sobota 07 sierpień 2004 16:04, jamal wrote:
> > > 	By the way, one of the consequence of versioning the API is
> > > that I tend to do most API changes in batches. The idea is that I want
> > > to minimise the number of API versions, because I have to test the
> > > tools and drivers with each of them, and I have finite time.
>
> There should really be no reason you have to change versions. It should
> be the last resort. You can use tricks like data structure augmentation
> and and new TLV types to go for a long time and still be backward (as
> well as forward) compatible. When you are no longer capable of doing
> these tricks, then it would make sense upping the version.
> There also should be rules for evolution reasons against having data
> structures which cross kernel/userspace from having things like
> lookatme[0] elements.

If I understand correctly how the new linux-abi headers are supposed to work - 
this is the way to go (meaning - new versions of linux-abi should not break 
old stuff if they don't have to).


-- 
In the year eighty five ten
God is gonna shake his mighty head
He'll either say,
"I'm pleased where man has been"
Or tear it down, and start again