File System Limitations

File System Limitations

[Fred]

Hi,

I need some basic info on File Size Limitations on filesystems such as ext2/3 
and ntfs.

I have looked on various search enfines and help sites but cannot find a list 
for comparison.

I need to store several 60GB files containing images.

Thanks


Fred

Re: File System Limitations

[Tony Hoyle]

In the ancient scrolls of Usenet, page
<01082316383301.12104@bits.linuxball>, "Fred" <fred@arkansaswebs.com>
spake thus:

> I need some basic info on File Size Limitations on filesystems such as
> ext2/3 and ntfs.

ext2/4 use are 64bit these days.  I guess signed so they'd have limit of
2TB per partition/file.

Tony

-- 
Microsoft - two out of three dead people who expressed a preference
said their coffins preferred it.

tmh@nothing-on.tv	http://www.nothing-on.tv

Re: File System Limitations

[Fred]

so why dos my filesystem have a 2 GB limit?
Must I specify a large block size or some such when i format?

i run 2.4.9 on redhat7.1 out of the box

Fred

 _________________________________________________ 
On Thursday 23 August 2001 05:58 pm, Tony Hoyle wrote:
> In the ancient scrolls of Usenet, page
> <01082316383301.12104@bits.linuxball>, "Fred" <fred@arkansaswebs.com>
>
> spake thus:
> > I need some basic info on File Size Limitations on filesystems such as
> > ext2/3 and ntfs.
>
> ext2/4 use are 64bit these days.  I guess signed so they'd have limit of
> 2TB per partition/file.
>
> Tony

Re: File System Limitations

[Tony Hoyle]

Fred wrote:
> so why dos my filesystem have a 2 GB limit?
> Must I specify a large block size or some such when i format?
> 
> i run 2.4.9 on redhat7.1 out of the box
> 
Does it?  Unless RH are using a seriously old glibc (which I doubt) 
there's no 2GB limit any more.

Some older applications don't work with it AFAIK... anything bundled 
with a modern distro shouldn't have any problems.

Tony

Re: File System Limitations

[Fred]

glibc-2.2.2-10

dd if=/dev/zero of=./tgb count=4000 bs=1M

created file of 2147483647 bytes

[root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
File size limit exceeded (core dumped)
[root@bits /a5]#

is glibc part of gcc? where do i find glibc?
(I've recently compiled gcc-3.00, but won't install cause it breaks kernel 
compilations).



TIA

Fred

 _________________________________________________ 
On Thursday 23 August 2001 06:18 pm, Tony Hoyle wrote:
> Fred wrote:
> > so why dos my filesystem have a 2 GB limit?
> > Must I specify a large block size or some such when i format?
> >
> > i run 2.4.9 on redhat7.1 out of the box
>
> Does it?  Unless RH are using a seriously old glibc (which I doubt)
> there's no 2GB limit any more.
>
> Some older applications don't work with it AFAIK... anything bundled
> with a modern distro shouldn't have any problems.
>
> Tony
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: File System Limitations

[Ignacio Vazquez-Abrams]

On Thu, 23 Aug 2001, Fred wrote:

> glibc-2.2.2-10
>
> dd if=/dev/zero of=./tgb count=4000 bs=1M
>
> created file of 2147483647 bytes
>
> [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> File size limit exceeded (core dumped)
> [root@bits /a5]#
>
> is glibc part of gcc? where do i find glibc?
> (I've recently compiled gcc-3.00, but won't install cause it breaks kernel
> compilations).
>
>
> TIA
>
> Fred

It looks as though dd has not been built with long file support. Most software
on a stock Red Hat install hasn't.

-- 
Ignacio Vazquez-Abrams  <ignacio@openservices.net>

Re: File System Limitations

[Alan Cox]

> glibc-2.2.2-10

Your C library is new enough

> [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> File size limit exceeded (core dumped)

But your dd program might not be

Alan

Re: File System Limitations

[Fred]

I tried dd (GNU fileutils) 4.0.36 (redhat 7.1)
and compiled and installed dd (fileutils) 4.1

still 
[root@bits /a5]# /usr/fileutils/bin/dd if=/dev/zero of=./tgb count=4000 bs=1M
File size limit exceeded (core dumped)
[root@bits /a5]# ls
total 2098436
      8 drwxr-xr-x    7 root     root         8192 Aug 23 22:07 .
   1120 -rwxr-xr-x    1 root     root      1146880 Aug 23 22:07 core
2097152 -rwxr-xr-x    1 root     root     2147483647 Aug 23 22:07 tgb

no errors in /var/log/messages


some system info
amd k6-II 500
256 MB ram
Ali 1541/3 chipset
WD 20GB 7200 rpm and WD 13GB 540 rpm on one cable
kernel-2.4.9
/a5 directory above is /dev/hda5 a vfat partition (is this the problem?)



TIA
Fred

 _________________________________________________ 
On Thursday 23 August 2001 06:52 pm, Alan Cox wrote:
> > glibc-2.2.2-10
>
> Your C library is new enough
>
> > [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> > File size limit exceeded (core dumped)
>
> But your dd program might not be
>
> Alan
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: File System Limitations

[Nicholas Knight]

On Thursday 23 August 2001 08:39 pm, Fred wrote:
> /a5 directory above is /dev/hda5 a vfat partition (is this the
> problem?)

that's exactly the problem, FAT32 doesn't support files over 2GB

Re: File System Limitations

[Doug McNaught]

Fred <fred@arkansaswebs.com> writes:

> /a5 directory above is /dev/hda5 a vfat partition (is this the problem?)

Almost certainly.

Try it on ext2 (if you have the space) and it should work.

-Doug
-- 
Free Dmitry Sklyarov! 
http://www.freesklyarov.org/ 

We will return to our regularly scheduled signature shortly.

Re: File System Limitations

[Ignacio Vazquez-Abrams]

On Thu, 23 Aug 2001, Fred wrote:

> I tried dd (GNU fileutils) 4.0.36 (redhat 7.1)
> and compiled and installed dd (fileutils) 4.1
>
> still
> [root@bits /a5]# /usr/fileutils/bin/dd if=/dev/zero of=./tgb count=4000 bs=1M
> File size limit exceeded (core dumped)
> [root@bits /a5]# ls
> total 2098436
>       8 drwxr-xr-x    7 root     root         8192 Aug 23 22:07 .
>    1120 -rwxr-xr-x    1 root     root      1146880 Aug 23 22:07 core
> 2097152 -rwxr-xr-x    1 root     root     2147483647 Aug 23 22:07 tgb
>
> no errors in /var/log/messages
>
>
> some system info
> amd k6-II 500
> 256 MB ram
> Ali 1541/3 chipset
> WD 20GB 7200 rpm and WD 13GB 540 rpm on one cable
> kernel-2.4.9
> /a5 directory above is /dev/hda5 a vfat partition (is this the problem?)
>
>
> TIA
> Fred

The problem is that dd was not compiled with -D_FILE_OFFSET_BITS=64.

-- 
Ignacio Vazquez-Abrams  <ignacio@openservices.net>

Re: File System Limitations

[Luigi Genoni]

I am just curious,
did you compiled your glibc against a  2.2 or 2.4 kernel headers???

Luigi

On Fri, 24 Aug 2001, Alan Cox wrote:

> > glibc-2.2.2-10
>
> Your C library is new enough
>
> > [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> > File size limit exceeded (core dumped)
>
> But your dd program might not be
>
> Alan
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: File System Limitations

[Brian]

Do you have a file size limit set in ulimit?
The last RedHat I installed had most of the limits set.

	-- Brian

On Thursday 23 August 2001 07:40 pm, Fred wrote:
> glibc-2.2.2-10
>
> dd if=/dev/zero of=./tgb count=4000 bs=1M
>
> created file of 2147483647 bytes
>
> [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> File size limit exceeded (core dumped)
> [root@bits /a5]#
>
> is glibc part of gcc? where do i find glibc?
> (I've recently compiled gcc-3.00, but won't install cause it breaks
> kernel compilations).
>
>
>
> TIA
>
> Fred
>
>  _________________________________________________
>
> On Thursday 23 August 2001 06:18 pm, Tony Hoyle wrote:
> > Fred wrote:
> > > so why dos my filesystem have a 2 GB limit?
> > > Must I specify a large block size or some such when i format?
> > >
> > > i run 2.4.9 on redhat7.1 out of the box
> >
> > Does it?  Unless RH are using a seriously old glibc (which I doubt)
> > there's no 2GB limit any more.
> >
> > Some older applications don't work with it AFAIK... anything bundled
> > with a modern distro shouldn't have any problems.
> >
> > Tony
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: File System Limitations

[Walter Hofmann]

On Thu, 23 Aug 2001, Fred wrote:

> glibc-2.2.2-10
> 
> dd if=/dev/zero of=./tgb count=4000 bs=1M
> 
> created file of 2147483647 bytes
> 
> [root@bits /a5]# dd if=/dev/zero of=./tgb count=4000 bs=1M
> File size limit exceeded (core dumped)

core dump???

My old dd terminated with an error in this case.

Do you have a file size limit set (ulimit -a)?

Walter

Re: File System Limitations

[Doug McNaught]

Tony Hoyle <tmh@nothing-on.tv> writes:

> Fred wrote:
> > so why dos my filesystem have a 2 GB limit?
> > Must I specify a large block size or some such when i format?
> > i run 2.4.9 on redhat7.1 out of the box
> 
> >
> 
> Does it?  Unless RH are using a seriously old glibc (which I doubt)
> there's no 2GB limit any more.

I just did:

[doug@abbadon workspace]$ dd if=/dev/zero of=foo bs=1024k count=3072
3072+0 records in
3072+0 records out
[doug@abbadon workspace]$ ls -l foo  
-rw-rw-r--    1 doug     doug     3221225472 Aug 23 20:01 foo
[doug@abbadon workspace]$ 

-Doug
-- 
Free Dmitry Sklyarov! 
http://www.freesklyarov.org/ 

We will return to our regularly scheduled signature shortly.

Re: File System Limitations

[Doug McNaught]

Doug McNaught <doug@wireboard.com> writes:

> Tony Hoyle <tmh@nothing-on.tv> writes:
> 
> > Does it?  Unless RH are using a seriously old glibc (which I doubt)
> > there's no 2GB limit any more.
> 
> I just did:
> 
> [doug@abbadon workspace]$ dd if=/dev/zero of=foo bs=1024k count=3072
> 3072+0 records in
> 3072+0 records out
> [doug@abbadon workspace]$ ls -l foo  
> -rw-rw-r--    1 doug     doug     3221225472 Aug 23 20:01 foo
> [doug@abbadon workspace]$ 

I meant to add that this is RH7.1, with updates applied, running
2.4.8-ac8. 

-Doug
-- 
Free Dmitry Sklyarov! 
http://www.freesklyarov.org/ 

We will return to our regularly scheduled signature shortly.