RE: Does e2fsprogs-1.26 work on mips?

RE: Does e2fsprogs-1.26 work on mips?

[Peter Hartley]

H J Lu wrote:
> I look at the glibc code. It uses a constant RLIM_INFINITY for a given
> arch. The user always passes (~0UL) to glibc on x86. glibc will check
> if the kernel supports the new getrlimit at the run time. If it
> doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I 
> don't see
> how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
> this unless glibc is misconfigureed or miscompiled.

It's not a question of which kernel glibc is compiled under, it's a question
of which version of the kernel headers (/usr/include/{linux,asm}) glibc is
compiled against.

A glibc, even the newest glibc, *compiled against 2.2 headers* cannot know
about the new getrlimit, so the run-time test cannot be compiled and is not
used. Such a glibc subsequently breaks fsck if run under a 2.4 kernel.

Recompile your glibc against 2.4 headers and you should get a glibc and fsck
that work if run under either a 2.2 or 2.4 kernel.

The necessary kernel patch to fix this mess is in the latest -pre-ac (thanks
Alan).

	Peter

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Mon, Mar 25, 2002 at 02:52:24AM -0800, Peter Hartley wrote:
> H J Lu wrote:
> > I look at the glibc code. It uses a constant RLIM_INFINITY for a given
> > arch. The user always passes (~0UL) to glibc on x86. glibc will check
> > if the kernel supports the new getrlimit at the run time. If it
> > doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I 
> > don't see
> > how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
> > this unless glibc is misconfigureed or miscompiled.
> 
> It's not a question of which kernel glibc is compiled under, it's a question
> of which version of the kernel headers (/usr/include/{linux,asm}) glibc is
> compiled against.
> 

What are you talking about? It doesn't matter which kernel header
is used. glibc doesn't even use /usr/include/asm/resource.h nor
should any user space applications.



H.J.

RE: Does e2fsprogs-1.26 work on mips?

[Peter Hartley]

H J Lu wrote:
> What are you talking about? It doesn't matter which kernel header
> is used. glibc doesn't even use /usr/include/asm/resource.h nor
> should any user space applications.

It's not about /usr/include/asm/resource.h, it's about
/usr/include/asm/unistd.h, where the syscall numbers are defined.

This is presumably what the "#ifdef __NR_ugetrlimit" in
sysdeps/unix/sysv/linux/i386/getrlimit.c is meant to be testing against --
nothing in the glibc-2.2.5 distribution itself defines that symbol. Surely a
Linux glibc doesn't compile without the target system's linux/* and asm/*
headers?

2.4's /usr/include/asm/unistd.h defines __NR_ugetrlimit but 2.2's doesn't.

	Peter

Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

I got

[root@localhost e2fsprogs-1.26]# ./e2fsck/e2fsck -f /dev/hda1
e2fsck 1.26 (3-Feb-2002)
Pass 1: Checking inodes, blocks, and sizes
File size limit exceeded

on Linux/mipsel. /dev/hda1 is a 7GB ext3 partition. e2fsprogs-1.23
works fine. Strace

open("/dev/hda1", O_RDWR|O_LARGEFILE)   = 4
fstat(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(3, 1), ...}) = 0
setrlimit(RLIMIT_FSIZE, {rlim_cur=-1, rlim_max=-1}) = 0
getrlimit(RLIMIT_FSIZE, {rlim_cur=-1, rlim_max=-1}) = 0
lseek(4, 1024, SEEK_SET)                = 1024
read(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
102
4
lseek(4, 4096, SEEK_SET)                = 4096
read(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
40
96
time(NULL)                              = 1016919001
lseek(4, 1072, SEEK_SET)                = 1072
write(4, "\331\363", 2)                 = 2
lseek(4, 1120, SEEK_SET)                = 1120
write(4, "\2\0", 2)                     = 2
lseek(4, 4096, SEEK_SET)                = 4096
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 134217728, SEEK_SET)           = 134217728
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 134221824, SEEK_SET)           = 134221824
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 402653184, SEEK_SET)           = 402653184
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 402657280, SEEK_SET)           = 402657280
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 671088640, SEEK_SET)           = 671088640
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 671092736, SEEK_SET)           = 671092736
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 939524096, SEEK_SET)           = 939524096
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 939528192, SEEK_SET)           = 939528192
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 1207959552, SEEK_SET)          = 1207959552
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 1207963648, SEEK_SET)          = 1207963648
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744072770027520, [3355443200], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
_llseek(4, 18446744072770031616, [3355447296], SEEK_SET) = 0
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744073038462976, [3623878656], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
_llseek(4, 18446744073038467072, [3623882752], SEEK_SET) = 0
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744071696285696, [6576668672], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
-1
 EFBIG (File too large)
--- SIGXFSZ (File size limit exceeded) ---
+++ killed by SIGXFSZ +++

Any ideas?


H.J.

Re: Does e2fsprogs-1.26 work on mips?

[Andrew Morton]

"H . J . Lu" wrote:
> 
> I got
> 
> [root@localhost e2fsprogs-1.26]# ./e2fsck/e2fsck -f /dev/hda1
> e2fsck 1.26 (3-Feb-2002)
> Pass 1: Checking inodes, blocks, and sizes
> File size limit exceeded
> 
> on Linux/mipsel. /dev/hda1 is a 7GB ext3 partition. e2fsprogs-1.23
> works fine. Strace
> 
> 

Common problem - it's due to internal API changes in resource
limits.  You need to ensure that your maximum file size
is set to `unlimited'.  Kernel is currently applying file
size limits to block devices (which is broken) and I
think e2fsprogs' attempt to set sile size limits to
RLIM_INFINITY gets broken by a consipiracy between the
kernel change and glibc headers.

Re: Does e2fsprogs-1.26 work on mips?

[H . J . Lu]

On Sat, Mar 23, 2002 at 11:03:30PM -0800, Andrew Morton wrote:
> "H . J . Lu" wrote:
> > 
> > ...
> > RLIM_INFINITY is not ((unsigned long)(~0UL)). Also you can't assume
> > the type of rlim.rlim_cur.
> > 
> > Here is a patch.
> > 
> 
> I suspect it's not right.
> 
> I don't pretend to understand the details, but they're
> messy.   See Ted's recent words at
> 
> 	http://www.uwsg.iu.edu/hypermail/linux/kernel/0203.2/0846.html
> 

I look at the glibc code. It uses a constant RLIM_INFINITY for a given
arch. The user always passes (~0UL) to glibc on x86. glibc will check
if the kernel supports the new getrlimit at the run time. If it
doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I don't see
how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
this unless glibc is misconfigureed or miscompiled.

> (Sorry - I should have dug that message out earlier).

The problem is not all arches use (~0UL) for RLIM_INFINITY.

# cd linux/include
# grep RLIM_INFINITY asm-*/resource.h | grep define
asm-alpha/resource.h:#define RLIM_INFINITY      0x7ffffffffffffffful
asm-arm/resource.h:#define RLIM_INFINITY        (~0UL)
asm-cris/resource.h:#define RLIM_INFINITY       (~0UL)
asm-i386/resource.h:#define RLIM_INFINITY       (~0UL)
asm-ia64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-m68k/resource.h:#define RLIM_INFINITY       (~0UL)
asm-mips64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-mips/resource.h:#define RLIM_INFINITY       0x7fffffffUL
asm-parisc/resource.h:#define RLIM_INFINITY   (~0UL)
asm-ppc/resource.h:#define RLIM_INFINITY        (~0UL)
asm-s390/resource.h:#define RLIM_INFINITY   (~0UL)
asm-s390x/resource.h:#define RLIM_INFINITY   (~0UL)
asm-sh/resource.h:#define RLIM_INFINITY (~0UL)
asm-sparc64/resource.h:#define RLIM_INFINITY    (~0UL)
asm-sparc/resource.h:#define RLIM_INFINITY      0x7fffffff

What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
nor alpha because of this. I don't think it works on sparc.

BTW, mips has

/*
 * SuS says limits have to be unsigned.
 * Which makes a ton more sense anyway.
 */
#define RLIM_INFINITY   0x7fffffffUL

It doesn't make any senes.


H.J.

Re: Does e2fsprogs-1.26 work on mips?

[Theodore Tso]

On Sun, Mar 24, 2002 at 01:28:19AM -0800, H . J . Lu wrote:
> 
> The problem is not all arches use (~0UL) for RLIM_INFINITY.
> 
> What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
> nor alpha because of this. I don't think it works on sparc.

Yeah, I forced the release of e2fsprogs 1.27 because of this, back on
March 8th.  That was my fault, and I fixed it as soon as I discovered
it.  (1.26 was released on Feb 3, and I released 1.27 on March 8th).

In e2fsprogs 1.27, I do the following:

#ifdef __linux__
#undef RLIM_INFINITY
#if (defined(__alpha__) || ((defined(__sparc__) || defined(__mips__)) && (SIZEOF_LONG == 4)))
#define RLIM_INFINITY	((unsigned long)(~0UL>>1))
#else
#define RLIM_INFINITY  (~0UL)
#endif

Basically because I can't depend on the RLIM_INFINITY being "right".
(Remember, I'm trying to make sure that e2fsprogs can compile on any
arbitrary glibc, and then run on any other-not-necessarily-the-same
glibc, which gets "challenging".)

The problem now is that some older glibcs are capping RLIM_INFINITY
with the older value, and so a combination of a new kernel, new
e2fsprogs, and old glibc results in problems.  My current feeling
about how to fix this is to bypass glibc altogether, and simply call
the setrlimit system call directly.  This is ugly-ugly-ugly, but I
can't see another way around this.

And just to be clear ---- although in the past I've been really
annoyed when glibc has made what I've considered to be arbitrary
changes which have screwed ABI, compile-time, or link-time
compatibility, and have spoken out against it --- in this particular
case, I consider the fault to be purely the fault of the kernel
developers, so there's no need having the glibc folks get all
defensive....

						- Ted

Re: Does e2fsprogs-1.26 work on mips?

[Andrew Morton]

Theodore Tso wrote:
> 
> And just to be clear ---- although in the past I've been really
> annoyed when glibc has made what I've considered to be arbitrary
> changes which have screwed ABI, compile-time, or link-time
> compatibility, and have spoken out against it --- in this particular
> case, I consider the fault to be purely the fault of the kernel
> developers, so there's no need having the glibc folks get all
> defensive....

So... Does the kernel need fixing? If so, what would you
recommend?

-

Re: Does e2fsprogs-1.26 work on mips?

[Theodore Tso]

On Sun, Mar 24, 2002 at 09:43:18PM -0800, Andrew Morton wrote:
> Theodore Tso wrote:
> > 
> > And just to be clear ---- although in the past I've been really
> > annoyed when glibc has made what I've considered to be arbitrary
> > changes which have screwed ABI, compile-time, or link-time
> > compatibility, and have spoken out against it --- in this particular
> > case, I consider the fault to be purely the fault of the kernel
> > developers, so there's no need having the glibc folks get all
> > defensive....
> 
> So... Does the kernel need fixing? If so, what would you
> recommend?

1) Created a new syscall for the unsinged setrlimit, not just for
getrlimit.  This should have been done from the very beginning, IMHO.

2) If the old value of RLIM_INFINITY is passed to the old setrlimit,
translate it to the new value of RLIM_INFINITY.  (This would not have
been strictly necessary of glibc wasn't playing RLIM_INIFITY capping
games; as it turns out, if you pass the "new" version of RLIM_INIFITY
to an "old" 2.2 kernel, the right thing happens.  So there really is
no need for glibc to cap the limit of RLIM_INFINITY to the old value.)

3)  RLIMIT_FILESIZE should not apply to block devices!!!

							- Ted

Re: Does e2fsprogs-1.26 work on mips?

[Paul Mackerras]

Theodore Tso writes:

> 3)  RLIMIT_FILESIZE should not apply to block devices!!!

Absolutely.

I would go further and say that it should only apply to writes to a
regular file that would extend the file past the filesize limit.  At
the moment the check in generic_file_write is simply whether the file
offset is greater than the limit, or would be greater than the limit
after the write.  This doesn't seem right to me.  If, for example, my
RLIMIT_FILESIZE is 1MB, and I have write access to an existing 100MB
file, I think I should be able to write anywhere in that file as long
as I don't try to extend it.

If we did that then the block device case would fall out, since you
can't extend block devices (not by writing past the end of them
anyway).

Paul.