File size limit exceeded with mkfs

File size limit exceeded with mkfs

[Jason Tackaberry]

Hi everyone.  Please CC any responses to me as I'm not subscribed to
this list.

I just installed a shiny new 80GB disk as primary slave and decided to
upgrade to 2.4.14+ext3 patch.  It appears that with this kernel, and
also with 2.4.15-pre7, when trying to mkfs partitions greater than 2GB,
I get "file size limit exceeded" and mkfs aborts.  I can successfully
mkfs partitions <= 2GB.

I do not have this problem with 2.4.7; all works well.  Wondering if the
difference was ext3, I tried compiling 2.4.15-pre7 without ext3 support
and the same problem occured.

The disk is

   hdb: 156301488 sectors (80026 MB) w/2048KiB Cache, CHS=9729/255/63

on a dual PIII-800 (SMP kernel).  Please let me know if I can supply
additional information.

Cheers,
Jason.

-- 
Academic Computing Support Specialist         Assistant Section Editor
Algoma University College                     http://linux.com/develop
Sault Ste. Marie, Ontario                 
705-949-2301 x330                                   Personal Home Page
http://www.auc.ca                                     http://sault.org

Re: File size limit exceeded with mkfs

[Andreas Dilger]

On Nov 20, 2001  11:02 -0500, Jason Tackaberry wrote:
> I just installed a shiny new 80GB disk as primary slave and decided to
> upgrade to 2.4.14+ext3 patch.  It appears that with this kernel, and
> also with 2.4.15-pre7, when trying to mkfs partitions greater than 2GB,
> I get "file size limit exceeded" and mkfs aborts.  I can successfully
> mkfs partitions <= 2GB.
> 
> I do not have this problem with 2.4.7; all works well.  Wondering if the
> difference was ext3, I tried compiling 2.4.15-pre7 without ext3 support
> and the same problem occured.

Several people have reported problems like this also.  What happens is
that if you are logged on as a user, then su to root, it will fail.  If
you log in directly as root, it will work.

I have looked through the 2.4.14 patch several times, but could not
find anything that might have caused this (I don't have any problems
with 2.4.13, but it may also have to do with my shell, glibc, or
whatever).

Can you please try some intermediate kernels (2.4.10 would be a good
start, because it had some major changes in this area, and then go
forward and back depending whether it works or not).

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: File size limit exceeded with mkfs

[Jason Tackaberry]

Hi Andreas,

On Tue, 2001-11-20 at 13:33, Andreas Dilger wrote:
> Several people have reported problems like this also.  What happens is
> that if you are logged on as a user, then su to root, it will fail.  If
> you log in directly as root, it will work.

Yep, this is indeed the case.

> Can you please try some intermediate kernels (2.4.10 would be a good
> start, because it had some major changes in this area, and then go
> forward and back depending whether it works or not).

2.4.10 does NOT work.
2.4.9 DOES work.

So clearly something happened in 2.4.10 which broke this.  Please let me
know if I can be of any more help.

Regards,
Jason.

Re: File size limit exceeded with mkfs

[Matti Aarnio]

On Tue, Nov 20, 2001 at 03:29:14PM -0500, Jason Tackaberry wrote:
> Hi Andreas,
> 
> On Tue, 2001-11-20 at 13:33, Andreas Dilger wrote:
... 
> > Can you please try some intermediate kernels (2.4.10 would be a good
> > start, because it had some major changes in this area, and then go
> > forward and back depending whether it works or not).
> 
> 2.4.10 does NOT work.
> 2.4.9 DOES work.

   And noting that  mkfs  and  fsck  operate on block DEVICE, no such
limits should be applied anyway.  Right ?

> So clearly something happened in 2.4.10 which broke this.  Please let me
> know if I can be of any more help.
> 
> Regards,
> Jason.

/Matti Aarnio

Re: File size limit exceeded with mkfs

[Neil Brown]

On Tuesday November 20, matti.aarnio@zmailer.org wrote:
> On Tue, Nov 20, 2001 at 03:29:14PM -0500, Jason Tackaberry wrote:
> > Hi Andreas,
> > 
> > On Tue, 2001-11-20 at 13:33, Andreas Dilger wrote:
> ... 
> > > Can you please try some intermediate kernels (2.4.10 would be a good
> > > start, because it had some major changes in this area, and then go
> > > forward and back depending whether it works or not).
> > 
> > 2.4.10 does NOT work.
> > 2.4.9 DOES work.
> 
>    And noting that  mkfs  and  fsck  operate on block DEVICE, no such
> limits should be applied anyway.  Right ?

No.  2.4.10 introduced blkdev-in-pagecache which made blockdev writes
succeptible to maximum file limits. (ulimit -f)

As a concession to backwards compatability, O_LARGEFILE is always used
when opening a block device.
Whoever ulimit still will stop you.

Some versions of libc think the file size limit is signed, so the max
you can set it to is 2^31-1.
But the kernel thinks it is unsigned, and assumes anything less than
2^32-1 is a real limit (2^32-1 is treated as "unlimited").

So if you are under the effect of default resource limits, then it
works.  If some parent process has tried to set the file limit to
"unlimited" it will have inadvertantly had exactly the reverse effect.

On my Debian/potato machines, I find that I hit the limit if I login
on the console, but if I ssh in as root, it works fine.

I think Debian/woody has a new enough libc that this isn't a problem.

NeilBrown

Re: File size limit exceeded with mkfs

[Andreas Dilger]

On Nov 20, 2001  15:29 -0500, Jason Tackaberry wrote:
> On Tue, 2001-11-20 at 13:33, Andreas Dilger wrote:
> > Several people have reported problems like this also.  What happens is
> > that if you are logged on as a user, then su to root, it will fail.  If
> > you log in directly as root, it will work.
> 
> Yep, this is indeed the case.
> 
> > Can you please try some intermediate kernels (2.4.10 would be a good
> > start, because it had some major changes in this area, and then go
> > forward and back depending whether it works or not).
> 
> 2.4.10 does NOT work.
> 2.4.9 DOES work.
> 
> So clearly something happened in 2.4.10 which broke this.  Please let me
> know if I can be of any more help.

That is unfortunate, since a lot of things changed in 2.4.10, so it will
make tracking the change hard.  Yet, I am running 2.4.13 and have no such
problems (well, at least I think not).  I don't have a spare partition >
2GB, but I can do the following without problems, which _should_ be the
same thing:

dd if=/dev/vgtest/lv of=/dev/vgtest/lv bs=4k skip=1100k seek=1100k count=1

(i.e. it reads a block and writes a block from > 4GB offset in the file,
and I can do this when logged in as a non-root user, which has write
access to the disk, and also when su'd to root, just in case).

Can you try the above test, just to confirm that it is equivalent to your
mkfs test?

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: File size limit exceeded with mkfs

[Mohammad A. Haque]

On Tuesday, November 20, 2001, at 06:35 PM, Andreas Dilger wrote:

> That is unfortunate, since a lot of things changed in 2.4.10, so it will
> make tracking the change hard.  Yet, I am running 2.4.13 and have no 
> such
> problems (well, at least I think not).  I don't have a spare partition >
> 2GB, but I can do the following without problems, which _should_ be the
> same thing:

could it be glibc 2.1.3 specific? i _think_ thats what I saw when the 
first round of reports came in (2.2.x users reported no problems).

--

=====================================================================
Mohammad A. Haque                              http://www.haque.net/
                                                mhaque@haque.net

   "Alcohol and calculus don't mix.             Developer/Project Lead
    Don't drink and derive." --Unknown          http://www.themes.org/
                                                batmanppc@themes.org
=====================================================================

Re: File size limit exceeded with mkfs

[Horst von Brand]

Andreas Dilger <adilger@turbolabs.com> said:
> On Nov 20, 2001  11:02 -0500, Jason Tackaberry wrote:
> > I just installed a shiny new 80GB disk as primary slave and decided to
> > upgrade to 2.4.14+ext3 patch.  It appears that with this kernel, and
> > also with 2.4.15-pre7, when trying to mkfs partitions greater than 2GB,
> > I get "file size limit exceeded" and mkfs aborts.  I can successfully
> > mkfs partitions <= 2GB.
> > 
> > I do not have this problem with 2.4.7; all works well.  Wondering if the
> > difference was ext3, I tried compiling 2.4.15-pre7 without ext3 support
> > and the same problem occured.
> 
> Several people have reported problems like this also.  What happens is
> that if you are logged on as a user, then su to root, it will fail.  If
> you log in directly as root, it will work.

This kind of differences (direct root login vs su) usualy are due to
different environment variable settings.

Just US$0.02
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

Re: File size limit exceeded with mkfs

[Peter Seiderer]

Horst von Brand wrote:
> This kind of differences (direct root login vs su) usualy are due to 
> different environment variable settings. 

Not in this case, it is because after 'su -' userland tries to
set the RLIMIT_FSIZE to unlimited with with the constant
0x7fffffff which is different from the one kernel uses internal
0xffffffff (see http://www.uwsg.indiana.edu/hypermail/linux/kernel/0111.1/0036.html and all next messages).

If this is more a gnulib or kernel mistake ? I don't know, but
the kernel uses sys_old_getrlimit and gives 0x7fffffff back to userland.

I think in this case the kernel should do the same in the other direction
(leafing a hole form 0x7fffffff to 0xffffffff where you could not set
the FSIZE).

Cheers,
 Peter