Re: Does e2fsprogs-1.26 work on mips?

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Sat, Mar 23, 2002 at 11:03:30PM -0800, Andrew Morton wrote:
> "H . J . Lu" wrote:
> > 
> > ...
> > RLIM_INFINITY is not ((unsigned long)(~0UL)). Also you can't assume
> > the type of rlim.rlim_cur.
> > 
> > Here is a patch.
> > 
> 
> I suspect it's not right.
> 
> I don't pretend to understand the details, but they're
> messy.   See Ted's recent words at
> 
> 	http://www.uwsg.iu.edu/hypermail/linux/kernel/0203.2/0846.html
> 

I look at the glibc code. It uses a constant RLIM_INFINITY for a given
arch. The user always passes (~0UL) to glibc on x86. glibc will check
if the kernel supports the new getrlimit at the run time. If it
doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I don't see
how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
this unless glibc is misconfigureed or miscompiled.

> (Sorry - I should have dug that message out earlier).

The problem is not all arches use (~0UL) for RLIM_INFINITY.

# cd linux/include
# grep RLIM_INFINITY asm-*/resource.h | grep define
asm-alpha/resource.h:#define RLIM_INFINITY      0x7ffffffffffffffful
asm-arm/resource.h:#define RLIM_INFINITY        (~0UL)
asm-cris/resource.h:#define RLIM_INFINITY       (~0UL)
asm-i386/resource.h:#define RLIM_INFINITY       (~0UL)
asm-ia64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-m68k/resource.h:#define RLIM_INFINITY       (~0UL)
asm-mips64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-mips/resource.h:#define RLIM_INFINITY       0x7fffffffUL
asm-parisc/resource.h:#define RLIM_INFINITY   (~0UL)
asm-ppc/resource.h:#define RLIM_INFINITY        (~0UL)
asm-s390/resource.h:#define RLIM_INFINITY   (~0UL)
asm-s390x/resource.h:#define RLIM_INFINITY   (~0UL)
asm-sh/resource.h:#define RLIM_INFINITY (~0UL)
asm-sparc64/resource.h:#define RLIM_INFINITY    (~0UL)
asm-sparc/resource.h:#define RLIM_INFINITY      0x7fffffff

What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
nor alpha because of this. I don't think it works on sparc.

BTW, mips has

/*
 * SuS says limits have to be unsigned.
 * Which makes a ton more sense anyway.
 */
#define RLIM_INFINITY   0x7fffffffUL

It doesn't make any senes.


H.J.

Re: Does e2fsprogs-1.26 work on mips?

[Theodore Tso]

On Sun, Mar 24, 2002 at 01:28:19AM -0800, H . J . Lu wrote:
> 
> The problem is not all arches use (~0UL) for RLIM_INFINITY.
> 
> What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
> nor alpha because of this. I don't think it works on sparc.

Yeah, I forced the release of e2fsprogs 1.27 because of this, back on
March 8th.  That was my fault, and I fixed it as soon as I discovered
it.  (1.26 was released on Feb 3, and I released 1.27 on March 8th).

In e2fsprogs 1.27, I do the following:

#ifdef __linux__
#undef RLIM_INFINITY
#if (defined(__alpha__) || ((defined(__sparc__) || defined(__mips__)) && (SIZEOF_LONG == 4)))
#define RLIM_INFINITY	((unsigned long)(~0UL>>1))
#else
#define RLIM_INFINITY  (~0UL)
#endif

Basically because I can't depend on the RLIM_INFINITY being "right".
(Remember, I'm trying to make sure that e2fsprogs can compile on any
arbitrary glibc, and then run on any other-not-necessarily-the-same
glibc, which gets "challenging".)

The problem now is that some older glibcs are capping RLIM_INFINITY
with the older value, and so a combination of a new kernel, new
e2fsprogs, and old glibc results in problems.  My current feeling
about how to fix this is to bypass glibc altogether, and simply call
the setrlimit system call directly.  This is ugly-ugly-ugly, but I
can't see another way around this.

And just to be clear ---- although in the past I've been really
annoyed when glibc has made what I've considered to be arbitrary
changes which have screwed ABI, compile-time, or link-time
compatibility, and have spoken out against it --- in this particular
case, I consider the fault to be purely the fault of the kernel
developers, so there's no need having the glibc folks get all
defensive....

						- Ted

Re: Does e2fsprogs-1.26 work on mips?

[Andrew Morton]

Theodore Tso wrote:
> 
> And just to be clear ---- although in the past I've been really
> annoyed when glibc has made what I've considered to be arbitrary
> changes which have screwed ABI, compile-time, or link-time
> compatibility, and have spoken out against it --- in this particular
> case, I consider the fault to be purely the fault of the kernel
> developers, so there's no need having the glibc folks get all
> defensive....

So... Does the kernel need fixing? If so, what would you
recommend?

-

RE: Does e2fsprogs-1.26 work on mips?

[Peter Hartley]

H J Lu wrote:
> I look at the glibc code. It uses a constant RLIM_INFINITY for a given
> arch. The user always passes (~0UL) to glibc on x86. glibc will check
> if the kernel supports the new getrlimit at the run time. If it
> doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I 
> don't see
> how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
> this unless glibc is misconfigureed or miscompiled.

It's not a question of which kernel glibc is compiled under, it's a question
of which version of the kernel headers (/usr/include/{linux,asm}) glibc is
compiled against.

A glibc, even the newest glibc, *compiled against 2.2 headers* cannot know
about the new getrlimit, so the run-time test cannot be compiled and is not
used. Such a glibc subsequently breaks fsck if run under a 2.4 kernel.

Recompile your glibc against 2.4 headers and you should get a glibc and fsck
that work if run under either a 2.2 or 2.4 kernel.

The necessary kernel patch to fix this mess is in the latest -pre-ac (thanks
Alan).

	Peter

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Mon, Mar 25, 2002 at 02:52:24AM -0800, Peter Hartley wrote:
> H J Lu wrote:
> > I look at the glibc code. It uses a constant RLIM_INFINITY for a given
> > arch. The user always passes (~0UL) to glibc on x86. glibc will check
> > if the kernel supports the new getrlimit at the run time. If it
> > doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I 
> > don't see
> > how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
> > this unless glibc is misconfigureed or miscompiled.
> 
> It's not a question of which kernel glibc is compiled under, it's a question
> of which version of the kernel headers (/usr/include/{linux,asm}) glibc is
> compiled against.
> 

What are you talking about? It doesn't matter which kernel header
is used. glibc doesn't even use /usr/include/asm/resource.h nor
should any user space applications.



H.J.

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Mon, Mar 25, 2002 at 12:31:59AM -0500, Theodore Tso wrote:
> On Sun, Mar 24, 2002 at 01:28:19AM -0800, H . J . Lu wrote:
> > 
> > The problem is not all arches use (~0UL) for RLIM_INFINITY.
> > 
> > What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
> > nor alpha because of this. I don't think it works on sparc.
> 
> Yeah, I forced the release of e2fsprogs 1.27 because of this, back on
> March 8th.  That was my fault, and I fixed it as soon as I discovered
> it.  (1.26 was released on Feb 3, and I released 1.27 on March 8th).
> 
> In e2fsprogs 1.27, I do the following:
> 
> #ifdef __linux__
> #undef RLIM_INFINITY
> #if (defined(__alpha__) || ((defined(__sparc__) || defined(__mips__)) && (SIZEOF_LONG == 4)))
> #define RLIM_INFINITY	((unsigned long)(~0UL>>1))
> #else
> #define RLIM_INFINITY  (~0UL)
> #endif
> 
> Basically because I can't depend on the RLIM_INFINITY being "right".
> (Remember, I'm trying to make sure that e2fsprogs can compile on any
> arbitrary glibc, and then run on any other-not-necessarily-the-same
> glibc, which gets "challenging".)
> 

The current glibc has no problem. You can just use RLIM_INFINITY
defined in GLIBC, not the kernel. The same static e2fsck binary will
work fine on both new and old kernels. But I don't know if you want
to pull the full setrlimit implemenation from glibc. I am enclosing
the i386 version here.



H.J.
---
/* Copyright (C) 1999, 2000 Free Software Foundation, Inc.
   This file is part of the GNU C Library.

   The GNU C Library is free software; you can redistribute it and/or
   modify it under the terms of the GNU Lesser General Public
   License as published by the Free Software Foundation; either
   version 2.1 of the License, or (at your option) any later version.

   The GNU C Library is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   Lesser General Public License for more details.

   You should have received a copy of the GNU Lesser General Public
   License along with the GNU C Library; if not, write to the Free
   Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
   02111-1307 USA.  */

#include <errno.h>
#include <sys/param.h>
#include <sys/resource.h>

#include <sysdep.h>
#include <sys/syscall.h>
#include <shlib-compat.h>
#include <bp-checks.h>

#include "kernel-features.h"

extern int __syscall_setrlimit (unsigned int resource,
				const struct rlimit *__unbounded rlimits);
extern int __syscall_ugetrlimit (unsigned int resource,
				 const struct rlimit *__unbounded rlimits);
extern int __new_setrlimit (enum __rlimit_resource resource,
			    const struct rlimit *__unboundedrlimits);

/* Linux 2.3.25 introduced a new system call since the types used for
   the limits are now unsigned.  */
#if defined __NR_ugetrlimit && !defined __ASSUME_NEW_GETRLIMIT_SYSCALL
extern int __have_no_new_getrlimit; /* from getrlimit.c */
#endif

int
__new_setrlimit (enum __rlimit_resource resource, const struct rlimit *rlimits)
{
#ifdef __ASSUME_NEW_GETRLIMIT_SYSCALL
  return INLINE_SYSCALL (setrlimit, 2, resource, CHECK_1 (rlimits));
#else
  struct rlimit rlimits_small;

# ifdef __NR_ugetrlimit
  if (__have_no_new_getrlimit == 0)
    {
      /* Check if the new ugetrlimit syscall exists.  We must do this
	 first because older kernels don't reject negative rlimit
	 values in setrlimit.  */
      int result = INLINE_SYSCALL (ugetrlimit, 2, resource, __ptrvalue (&rlimits_small));
      if (result != -1 || errno != ENOSYS)
	/* The syscall exists.  */
	__have_no_new_getrlimit = -1;
      else
	/* The syscall does not exist.  */
	__have_no_new_getrlimit = 1;
    }
  if (__have_no_new_getrlimit < 0)
    return INLINE_SYSCALL (setrlimit, 2, resource, CHECK_1 (rlimits));
# endif

  /* We might have to correct the limits values.  Since the old values
     were signed the new values might be too large.  */
  rlimits_small.rlim_cur = MIN ((unsigned long int) rlimits->rlim_cur,
				RLIM_INFINITY >> 1);
  rlimits_small.rlim_max = MIN ((unsigned long int) rlimits->rlim_max,
				RLIM_INFINITY >> 1);

  /* Use the adjusted values.  */
  return INLINE_SYSCALL (setrlimit, 2, resource, __ptrvalue (&rlimits_small));
#endif
}

weak_alias (__new_setrlimit, __setrlimit);
versioned_symbol (libc, __new_setrlimit, setrlimit, GLIBC_2_2);

RE: Does e2fsprogs-1.26 work on mips?

[Peter Hartley]

H J Lu wrote:
> What are you talking about? It doesn't matter which kernel header
> is used. glibc doesn't even use /usr/include/asm/resource.h nor
> should any user space applications.

It's not about /usr/include/asm/resource.h, it's about
/usr/include/asm/unistd.h, where the syscall numbers are defined.

This is presumably what the "#ifdef __NR_ugetrlimit" in
sysdeps/unix/sysv/linux/i386/getrlimit.c is meant to be testing against --
nothing in the glibc-2.2.5 distribution itself defines that symbol. Surely a
Linux glibc doesn't compile without the target system's linux/* and asm/*
headers?

2.4's /usr/include/asm/unistd.h defines __NR_ugetrlimit but 2.2's doesn't.

	Peter

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Mon, Mar 25, 2002 at 11:00:05AM -0800, Peter Hartley wrote:
> H J Lu wrote:
> > What are you talking about? It doesn't matter which kernel header
> > is used. glibc doesn't even use /usr/include/asm/resource.h nor
> > should any user space applications.
> 
> It's not about /usr/include/asm/resource.h, it's about
> /usr/include/asm/unistd.h, where the syscall numbers are defined.
> 
> This is presumably what the "#ifdef __NR_ugetrlimit" in
> sysdeps/unix/sysv/linux/i386/getrlimit.c is meant to be testing against --
> nothing in the glibc-2.2.5 distribution itself defines that symbol. Surely a
> Linux glibc doesn't compile without the target system's linux/* and asm/*
> headers?
> 
> 2.4's /usr/include/asm/unistd.h defines __NR_ugetrlimit but 2.2's doesn't.
> 

I see. I think glibc should either require 2.4 header files under
<asm/*.h> and <linux/*.h>, or define __NR_ugetrlimit if it is not
defined.


H.J.

Re: Does e2fsprogs-1.26 work on mips?

[Theodore Tso]

On Sun, Mar 24, 2002 at 09:43:18PM -0800, Andrew Morton wrote:
> Theodore Tso wrote:
> > 
> > And just to be clear ---- although in the past I've been really
> > annoyed when glibc has made what I've considered to be arbitrary
> > changes which have screwed ABI, compile-time, or link-time
> > compatibility, and have spoken out against it --- in this particular
> > case, I consider the fault to be purely the fault of the kernel
> > developers, so there's no need having the glibc folks get all
> > defensive....
> 
> So... Does the kernel need fixing? If so, what would you
> recommend?

1) Created a new syscall for the unsinged setrlimit, not just for
getrlimit.  This should have been done from the very beginning, IMHO.

2) If the old value of RLIM_INFINITY is passed to the old setrlimit,
translate it to the new value of RLIM_INFINITY.  (This would not have
been strictly necessary of glibc wasn't playing RLIM_INIFITY capping
games; as it turns out, if you pass the "new" version of RLIM_INIFITY
to an "old" 2.2 kernel, the right thing happens.  So there really is
no need for glibc to cap the limit of RLIM_INFINITY to the old value.)

3)  RLIMIT_FILESIZE should not apply to block devices!!!

							- Ted

Re: Does e2fsprogs-1.26 work on mips?

[Paul Mackerras]

Theodore Tso writes:

> 3)  RLIMIT_FILESIZE should not apply to block devices!!!

Absolutely.

I would go further and say that it should only apply to writes to a
regular file that would extend the file past the filesize limit.  At
the moment the check in generic_file_write is simply whether the file
offset is greater than the limit, or would be greater than the limit
after the write.  This doesn't seem right to me.  If, for example, my
RLIMIT_FILESIZE is 1MB, and I have write access to an existing 100MB
file, I think I should be able to write anywhere in that file as long
as I don't try to extend it.

If we did that then the block device case would fall out, since you
can't extend block devices (not by writing past the end of them
anyway).

Paul.