[PATCH] sysctl: Undeprecate sys_sysctl

[PATCH] sysctl: Undeprecate sys_sysctl

[Eric W. Biederman]

The basic issue is that despite have been deprecated and warned about
as a very bad thing in the man pages since its inception there are a
few real users of sys_sysctl.  It was my assumption that because
sysctl had been deprecated for all of 2.6 there would be no user space
users by this point, so I initially gave sys_sysctl a very short
deprecation period.

Now that I know there are a few real users the only sane way to
proceed with deprecation is to push the time limit out to a year or
two work and work with distributions that have big testing pools like
fedora core to find these last remaining users.

Which means that the sys_sysctl interface needs to be maintained in
the meantime. 

Since I have provided a technical measure that allows us to add new
sysctl entries without reserving more binary numbers I believe that is
enough to fix the sys_sysctl binary interface maintenance problems,
because there is no longer a need to change the binary interface at
all.

Since the sys_sysctl implementation needs to stay around for a while
and the worst of the maintenance issues that caused us to occasionally
break the ABI have been addressed I don't see any advantage in
continuing with the removal of sys_sysctl.

So instead of merely increasing the deprecation period this patch
removes the deprecation of sys_sysctl and modifies the kernel to
compile the code in by default.

With committing to maintain sys_sysctl we get all of the advantages of
a fast interface for anything that needs it.  Currently sys_sysctl is
about 5x faster than /proc/sys, for the same string data.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
 Documentation/feature-removal-schedule.txt |   12 ------------
 init/Kconfig                               |    2 +-
 2 files changed, 1 insertions(+), 13 deletions(-)

diff --git a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
index 1ac3c74..d52c4aa 100644
--- a/Documentation/feature-removal-schedule.txt
+++ b/Documentation/feature-removal-schedule.txt
@@ -53,18 +53,6 @@ Who:	Mauro Carvalho Chehab <mchehab@brtu
 
 ---------------------------
 
-What:	sys_sysctl
-When:	January 2007
-Why:	The same information is available through /proc/sys and that is the
-	interface user space prefers to use. And there do not appear to be
-	any existing user in user space of sys_sysctl.  The additional
-	maintenance overhead of keeping a set of binary names gets
-	in the way of doing a good job of maintaining this interface.
-
-Who:	Eric Biederman <ebiederm@xmission.com>
-
----------------------------
-
 What:	PCMCIA control ioctl (needed for pcmcia-cs [cardmgr, cardctl])
 When:	November 2005
 Files:	drivers/pcmcia/: pcmcia_ioctl.c
diff --git a/init/Kconfig b/init/Kconfig
index c8b2624..e85e554 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -304,7 +304,7 @@ config UID16
 
 config SYSCTL_SYSCALL
 	bool "Sysctl syscall support" if EMBEDDED
-	default n
+	default y
 	select SYSCTL
 	---help---
 	  Enable the deprecated sysctl system call.  sys_sysctl uses
-- 
1.4.2.rc3.g7e18e-dirty

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Jesper Juhl]

On 08/11/06, Eric W. Biederman <ebiederm@xmission.com> wrote:
>
[...]
>
> Since the sys_sysctl implementation needs to stay around for a while
> and the worst of the maintenance issues that caused us to occasionally
> break the ABI have been addressed I don't see any advantage in
> continuing with the removal of sys_sysctl.
>
> So instead of merely increasing the deprecation period this patch
> removes the deprecation of sys_sysctl and modifies the kernel to
> compile the code in by default.
>
[...]
>
>  config SYSCTL_SYSCALL
>         bool "Sysctl syscall support" if EMBEDDED
> -       default n
> +       default y
>         select SYSCTL
>         ---help---
>           Enable the deprecated sysctl system call.  sys_sysctl uses

Perhaps you should also change the help text here to not say "the
deprecated sysctl system call". Perhaps change to something like :

"Deselect this option, to not build sysctl suport into the kernel, if
you are sure you won't be running any applications that depend on the
sysctl system call.
If unsure, say Y."

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

[PATCH] sysctl: Undeprecate sys_sysctl (take 2)

[Eric W. Biederman]

The basic issue is that despite have been ``deprecated'' and
warned about as a very bad thing in the man pages since it's
inception there are a few real users of sys_sysctl.  It was
my assumption that because sysctl had been ``deprecated''
for all of 2.6 there would be no user space users by this
point, so I initially gave sys_sysctl a very short deprecation
period.

Now that I know there are a few real users the only sane way
to proceed with deprecation is to push the time limit out
to a year or two work and work with distributions that
have big testing pools like fedora core to find these last
remaining users.

Which means that the sys_sysctl interface needs to be maintained
in the meantime.

Since I have provided a technical measure that allows us
to add new sysctl entries without reserving more binary
numbers I believe that is enough to fix the sys_sysctl
binary interface maintenance problems (I.e. don't grow it
any more).

Since the sys_sysctl implementation needs to stay around for
a while and the worst of the maintenance issues that caused us
to occassionally break the ABI have been addressed I don't
see any advantage in continuing with the removal of sys_sysctl.

With committing to maintain sys_sysctl we get all of the
advantages of a fast interface for anything that needs
it.  Currently sys_sysctl is about 5x faster than /proc/sys,
for the same string data.

The Kconfig help text is now also fixed.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
 Documentation/feature-removal-schedule.txt |   12 ------------
 init/Kconfig                               |   19 +++++++++----------
 2 files changed, 9 insertions(+), 22 deletions(-)

diff --git a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
index 1ac3c74..d52c4aa 100644
--- a/Documentation/feature-removal-schedule.txt
+++ b/Documentation/feature-removal-schedule.txt
@@ -53,18 +53,6 @@ Who:	Mauro Carvalho Chehab <mchehab@brtu
 
 ---------------------------
 
-What:	sys_sysctl
-When:	January 2007
-Why:	The same information is available through /proc/sys and that is the
-	interface user space prefers to use. And there do not appear to be
-	any existing user in user space of sys_sysctl.  The additional
-	maintenance overhead of keeping a set of binary names gets
-	in the way of doing a good job of maintaining this interface.
-
-Who:	Eric Biederman <ebiederm@xmission.com>
-
----------------------------
-
 What:	PCMCIA control ioctl (needed for pcmcia-cs [cardmgr, cardctl])
 When:	November 2005
 Files:	drivers/pcmcia/: pcmcia_ioctl.c
diff --git a/init/Kconfig b/init/Kconfig
index c8b2624..ffe945a 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -304,20 +304,19 @@ config UID16
 
 config SYSCTL_SYSCALL
 	bool "Sysctl syscall support" if EMBEDDED
-	default n
+	default y
 	select SYSCTL
 	---help---
-	  Enable the deprecated sysctl system call.  sys_sysctl uses
-	  binary paths that have been found to be a major pain to maintain
-	  and use.  The interface in /proc/sys is now the primary and what
-	  everyone uses.
+	  sys_sysctl uses binary paths that have been found challenging
+	  to properly maintain and use.  The interface in /proc/sys
+	  using paths with ascii names is now the primary path to this
+	  information. 
 
-	  Nothing has been using the binary sysctl interface for some
-	  time now so nothing should break if you disable sysctl syscall
-	  support, and your kernel will get marginally smaller.
+	  Almost nothing using the binary sysctl interface so if you are
+	  trying to save some space it is probably safe to disable this,
+	  making your kernel marginally smaller.
 
-	  Unless you have an application that uses the sys_sysctl interface
- 	  you should probably say N here.
+	  If unsure say Y here.
 
 config KALLSYMS
 	 bool "Load all symbols for debugging/kksymoops" if EMBEDDED
-- 
1.4.2.rc3.g7e18e-dirty

Re: [PATCH] sysctl: Undeprecate sys_sysctl (take 2)

[Andi Kleen]

On Wednesday 08 November 2006 20:58, Eric W. Biederman wrote:
> 
> The basic issue is that despite have been ``deprecated'' and
> warned about as a very bad thing in the man pages since it's
> inception there are a few real users of sys_sysctl. 

But they only seem to use a small number of actually used with
sysctl(2) sysctls.
I still think just maintaining a conversion table for 
those is the right thing to do.

The important part really is to get rid of the crufty 
old infrastructure internally.

-Andi

Re: [PATCH] sysctl: Undeprecate sys_sysctl (take 2)

[Eric W. Biederman]

Andi Kleen <ak@suse.de> writes:

> On Wednesday 08 November 2006 20:58, Eric W. Biederman wrote:
>> 
>> The basic issue is that despite have been ``deprecated'' and
>> warned about as a very bad thing in the man pages since it's
>> inception there are a few real users of sys_sysctl. 
>
> But they only seem to use a small number of actually used with
> sysctl(2) sysctls.
> I still think just maintaining a conversion table for 
> those is the right thing to do.

I don't know.  Every distinct user of the binary sysctl interface
used a different entry.  So the fact that there are a small number of
programs and thus a small number of sysctls used I agree with.  I do
not agree with the conclusion that we can predict the set of binary
sysctl that are in use.  We do not get good enough feedback from
the user community.

I don't have a problem with the principle of a conversion table
if it meant that we would never add any additional binary sysctls.

> The important part really is to get rid of the crufty 
> old infrastructure internally.

There is certainly some cleanup we could do there.  For absolutely
simple values sysctl isn't too bad.  But generated values are
horrible.

Certainly some of the parts are crufty and harder to use then
they should be.  The /proc side actually seems worse than the
binary side.

That said I have some plans on attacking sysctl in a very practical
sense, because it is required to implement multiple namespaces.

Eric

Re: [PATCH] sysctl: Undeprecate sys_sysctl (take 2)

[H. Peter Anvin]

Eric W. Biederman wrote:
> Andi Kleen <ak@suse.de> writes:
> 
>> On Wednesday 08 November 2006 20:58, Eric W. Biederman wrote:
>>> The basic issue is that despite have been ``deprecated'' and
>>> warned about as a very bad thing in the man pages since it's
>>> inception there are a few real users of sys_sysctl. 
>> But they only seem to use a small number of actually used with
>> sysctl(2) sysctls.
>> I still think just maintaining a conversion table for 
>> those is the right thing to do.
> 
> I don't know.  Every distinct user of the binary sysctl interface
> used a different entry.  So the fact that there are a small number of
> programs and thus a small number of sysctls used I agree with.  I do
> not agree with the conclusion that we can predict the set of binary
> sysctl that are in use.  We do not get good enough feedback from
> the user community.
> 
> I don't have a problem with the principle of a conversion table
> if it meant that we would never add any additional binary sysctls.
>

Okay, my opinion now...

I think we should change the sysctl system so most sysctls simply aren't 
accessible through the binary interface.  The rest of them should be 
documented in one place, preferrably machine-readable.

However, I think having the binary sysctls available as a limited last 
resort is better than adding ad hoc system calls all over the place, 
like sys_mips.

	-hpa

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Alan Cox]

On Wed, Nov 08, 2006 at 12:00:10PM -0700, Eric W. Biederman wrote:
> Now that I know there are a few real users the only sane way to
> proceed with deprecation is to push the time limit out to a year or
> two work and work with distributions that have big testing pools like
> fedora core to find these last remaining users.

Some early boot code needs to know the kernel version and
it needs to do it before /proc is mounted and potentially in order
to run mount. In places it has its role but only in places.

> diff --git a/init/Kconfig b/init/Kconfig
> index c8b2624..e85e554 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -304,7 +304,7 @@ config UID16
>  
>  config SYSCTL_SYSCALL
>  	bool "Sysctl syscall support" if EMBEDDED
> -	default n
> +	default y
>  	select SYSCTL
>  	---help---
>  	  Enable the deprecated sysctl system call.  sys_sysctl uses
> -- 
> 1.4.2.rc3.g7e18e-dirty

Acked-by: Alan Cox <alan@redhat.com>

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Arjan van de Ven]

On Wed, 2006-11-08 at 15:49 -0500, Alan Cox wrote:
> On Wed, Nov 08, 2006 at 12:00:10PM -0700, Eric W. Biederman wrote:
> > Now that I know there are a few real users the only sane way to
> > proceed with deprecation is to push the time limit out to a year or
> > two work and work with distributions that have big testing pools like
> > fedora core to find these last remaining users.
> 
> Some early boot code needs to know the kernel version and
> it needs to do it before /proc is mounted and potentially in order
> to run mount. In places it has its role but only in places.

isn't kernel version passed as AT vector nowadays?

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[H. Peter Anvin]

Alan Cox wrote:
> On Wed, Nov 08, 2006 at 12:00:10PM -0700, Eric W. Biederman wrote:
>> Now that I know there are a few real users the only sane way to
>> proceed with deprecation is to push the time limit out to a year or
>> two work and work with distributions that have big testing pools like
>> fedora core to find these last remaining users.
> 
> Some early boot code needs to know the kernel version and
> it needs to do it before /proc is mounted and potentially in order
> to run mount. In places it has its role but only in places.

uname() provides the kernel version.

	-hpa

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Alistair John Strachan]

On Wednesday 08 November 2006 19:00, you wrote:
> The basic issue is that despite have been deprecated and warned about
> as a very bad thing in the man pages since its inception there are a
> few real users of sys_sysctl.  It was my assumption that because
> sysctl had been deprecated for all of 2.6 there would be no user space
> users by this point, so I initially gave sys_sysctl a very short
> deprecation period.
>
> Now that I know there are a few real users the only sane way to
> proceed with deprecation is to push the time limit out to a year or
> two work and work with distributions that have big testing pools like
> fedora core to find these last remaining users.

Eric, do you have a list of the remaining users? It'd be good to know for 
people using Linux in an embedded environment, where they may want to switch 
off the option, but only if it doesn't break their userspace.

-- 
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Eric W. Biederman]

Alistair John Strachan <s0348365@sms.ed.ac.uk> writes:

> On Wednesday 08 November 2006 19:00, you wrote:
>> The basic issue is that despite have been deprecated and warned about
>> as a very bad thing in the man pages since its inception there are a
>> few real users of sys_sysctl.  It was my assumption that because
>> sysctl had been deprecated for all of 2.6 there would be no user space
>> users by this point, so I initially gave sys_sysctl a very short
>> deprecation period.
>>
>> Now that I know there are a few real users the only sane way to
>> proceed with deprecation is to push the time limit out to a year or
>> two work and work with distributions that have big testing pools like
>> fedora core to find these last remaining users.
>
> Eric, do you have a list of the remaining users? It'd be good to know for 
> people using Linux in an embedded environment, where they may want to switch 
> off the option, but only if it doesn't break their userspace.

They are very very few.  The ones I recall are kudzu, radvd, and
libpthreads (which doesn't care).  

There is a thread a month or so ago about this where I did a request
for testers, that listed all of the users we could find.

The reality is that I don't think kernel developers can seriously find
them.

If someone actually wants to kill sys_sysctl more power to them.  As
long as we don't add more binary numbers I think it is actually easier
to support it than to find those weird users and remove it.

Eric

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Russell King]

On Thu, Nov 09, 2006 at 10:21:13PM -0700, Eric W. Biederman wrote:
> Alistair John Strachan <s0348365@sms.ed.ac.uk> writes:
> > Eric, do you have a list of the remaining users? It'd be good to know for 
> > people using Linux in an embedded environment, where they may want to switch 
> > off the option, but only if it doesn't break their userspace.
> 
> They are very very few.  The ones I recall are kudzu, radvd, and
> libpthreads (which doesn't care).  

Let's not forget that on ARM it's used to get the MMIO region which
is used for PIO emulation by glibc.  Without it, glibc can't detect
where this region is.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Eric W. Biederman]

Russell King <rmk+lkml@arm.linux.org.uk> writes:

> On Thu, Nov 09, 2006 at 10:21:13PM -0700, Eric W. Biederman wrote:
>> Alistair John Strachan <s0348365@sms.ed.ac.uk> writes:
>> > Eric, do you have a list of the remaining users? It'd be good to know for 
>> > people using Linux in an embedded environment, where they may want to switch
>> > off the option, but only if it doesn't break their userspace.
>> 
>> They are very very few.  The ones I recall are kudzu, radvd, and
>> libpthreads (which doesn't care).  
>
> Let's not forget that on ARM it's used to get the MMIO region which
> is used for PIO emulation by glibc.  Without it, glibc can't detect
> where this region is.

Is that on all subarches or just some of them?

But given the scarcity of the list this is certainly one of the
significant known users.

Eric

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Russell King]

On Fri, Nov 10, 2006 at 01:13:16AM -0700, Eric W. Biederman wrote:
> Russell King <rmk+lkml@arm.linux.org.uk> writes:
> > Let's not forget that on ARM it's used to get the MMIO region which
> > is used for PIO emulation by glibc.  Without it, glibc can't detect
> > where this region is.
> 
> Is that on all subarches or just some of them?

No idea - I don't have that sort of information.  Those which are PCI
based have the potential to use it if some userspace programmer wants
to use the ioperm/inb/outb etc which glibc provides.

I have had the occasional bug report that these functions haven't
worked when we haven't been exporting the right info via sysctl, so
they do seem to get used.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: [PATCH] sysctl: Undeprecate sys_sysctl

[Alistair John Strachan]

On Friday 10 November 2006 05:21, Eric W. Biederman wrote:
> Alistair John Strachan <s0348365@sms.ed.ac.uk> writes:
> > On Wednesday 08 November 2006 19:00, you wrote:
> >> The basic issue is that despite have been deprecated and warned about
> >> as a very bad thing in the man pages since its inception there are a
> >> few real users of sys_sysctl.  It was my assumption that because
> >> sysctl had been deprecated for all of 2.6 there would be no user space
> >> users by this point, so I initially gave sys_sysctl a very short
> >> deprecation period.
> >>
> >> Now that I know there are a few real users the only sane way to
> >> proceed with deprecation is to push the time limit out to a year or
> >> two work and work with distributions that have big testing pools like
> >> fedora core to find these last remaining users.
> >
> > Eric, do you have a list of the remaining users? It'd be good to know for
> > people using Linux in an embedded environment, where they may want to
> > switch off the option, but only if it doesn't break their userspace.
>
> They are very very few.  The ones I recall are kudzu, radvd, and
> libpthreads (which doesn't care).

Neither do new versions of radvd, from what I can see.

radvd.c/check_ip6_forwarding():

#ifdef __linux__
	fp = fopen(PROC_SYS_IP6_FORWARDING, "r");
	if (fp) {
		fscanf(fp, "%d", &value);
		fclose(fp);
	}
	else
		flog(LOG_DEBUG, "Correct IPv6 forwarding procfs entry not found, "
	                       "perhaps the procfs is disabled, "
	                        "or the kernel interface has changed?");
#endif /* __linux__ */

	if (!fp && sysctl(forw_sysctl, sizeof(forw_sysctl)/sizeof(forw_sysctl[0]),
	    &value, &size, NULL, 0) < 0) {
		flog(LOG_DEBUG, "Correct IPv6 forwarding sysctl branch not found, "
			"perhaps the kernel interface has changed?");
		return(0);	/* this is of advisory value only */
	}

Maybe you should just hold out until everybody fixes their programs ;-)

-- 
Cheers,
Alistair.

Final year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.