Re: Filesize limit on SMBFS

Re: Filesize limit on SMBFS

[Tyler BIRD]

Ext2 Filesystems I believe have the limit of 2 GB.  Ext3 Extends that Limit to something??
Try making the ext3 filesystem partitions and sharing those.  
I don't know limits on FAT32 or any other filesystem you can share

Tyler

>>> Petr Tite(ra <P.Titera@century.cz> 11/22/01 02:10AM >>>
Hello,

    is maximum file size on SMBFS really 2GB? I cannot create file 
bigger than that.

Petr Titera
P.Titera@century.cz 


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org 
More majordomo info at  http://vger.kernel.org/majordomo-info.html 
Please read the FAQ at  http://www.tux.org/lkml/

Re: Filesize limit on SMBFS

[Marcelo Borges Ribeiro]

This limit is a kernel´s limit not a file system´s limit. Even vfat has a
limitation of 2GB under linux. I thought with kernel 2.4.x this will be
over.
----- Original Message -----
From: "Tyler BIRD" <birdty@uvsc.edu>
To: <P.Titera@century.cz>; <linux-kernel@vger.kernel.org>
Sent: Thursday, November 22, 2001 4:53 PM
Subject: Re: Filesize limit on SMBFS


> Ext2 Filesystems I believe have the limit of 2 GB.  Ext3 Extends that
Limit to something??
> Try making the ext3 filesystem partitions and sharing those.
> I don't know limits on FAT32 or any other filesystem you can share
>
> Tyler
>
> >>> Petr Tite(ra <P.Titera@century.cz> 11/22/01 02:10AM >>>
> Hello,
>
>     is maximum file size on SMBFS really 2GB? I cannot create file
> bigger than that.
>
> Petr Titera
> P.Titera@century.cz
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Filesize limit on SMBFS

[Andreas Dilger]

On Nov 22, 2001  17:23 -0200, Marcelo Borges Ribeiro wrote:
> This limit is a kernel´s limit not a file system´s limit. Even vfat has a
> limitation of 2GB under linux. I thought with kernel 2.4.x this will be
> over.

Totally incorrect.  2.4 allows files larger than 2GB, and with a patch,
you can do this on 2.2 as well.  If you are having problems with a 2GB
limit, then either your shell, libc, or tools is causing the problem.

VFAT does have a 2GB limit, AFAIK, but I could be wrong.

> ----- Original Message -----
> From: "Tyler BIRD" <birdty@uvsc.edu>
> > Ext2 Filesystems I believe have the limit of 2 GB.  Ext3 Extends that
> > Limit to something??

No, the ext2 and ext3 limits are exactly the same, about 4TB right
now, but they would be larger with a bit of bug fixing (up to 16TB).
Note that the kernel has a limit of 2TB for a single device.

> >     is maximum file size on SMBFS really 2GB? I cannot create file
> > bigger than that.

As for SMBFS, I don't know, but it can obviously not be larger than the
limit on the server.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Filesize limit on SMBFS

[Jeff Chua]

On Thu, 22 Nov 2001, Andreas Dilger wrote:

> VFAT does have a 2GB limit, AFAIK, but I could be wrong.

Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.

I'm using 3GB for VFAT partition.

Jeff.

Re: Filesize limit on SMBFS

[Marcelo Borges Ribeiro]

I have fat32 partition, but the problem isn´t the size of partition it is
8GB. The problem is that if you want to
create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2
to a reiserfs) to an available fat32 space, in my case the backup size is
allways 2GB and when I tried to extract back I saw "unexpected end of file".
So I thought it was that famous kernel limitation of 2GB under any kind of
partition, but i was informed that fat has this limitation too. So the
kernel may suport files bigger than 2GB (I really don´t know, I just know
that in my case with fat32 it did not and I saw this too with some oracle
databases that could not be used when they grow and reach 2GB, may be a
library problem too).

That´s all.
----- Original Message -----
From: "Jeff Chua" <jchua@fedex.com>
To: "Andreas Dilger" <adilger@turbolabs.com>
Cc: "Marcelo Borges Ribeiro" <marcelo@datacom-telematica.com.br>; "Tyler
BIRD" <birdty@uvsc.edu>; <linux-kernel@vger.kernel.org>
Sent: Friday, November 23, 2001 12:35 AM
Subject: Re: Filesize limit on SMBFS


> On Thu, 22 Nov 2001, Andreas Dilger wrote:
>
> > VFAT does have a 2GB limit, AFAIK, but I could be wrong.
>
> Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.
>
> I'm using 3GB for VFAT partition.
>
> Jeff.
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Filesize limit on SMBFS

[Jeff Chua]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 1946 bytes --]


You may want to try using glibc 2.2.x, but then you may fail to compile
many other codes. I tried it once, and managed to get >2GB file, but had
to revert back to 2.1.3 because 2.2.x broke compiling other codes.
That was a few months back.


Thanks,
Jeff
[ jchua@fedex.com ]

On Fri, 23 Nov 2001, Marcelo Borges Ribeiro wrote:

> I have fat32 partition, but the problem isn´t the size of partition it is
> 8GB. The problem is that if you want to
> create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2
> to a reiserfs) to an available fat32 space, in my case the backup size is
> allways 2GB and when I tried to extract back I saw "unexpected end of file".
> So I thought it was that famous kernel limitation of 2GB under any kind of
> partition, but i was informed that fat has this limitation too. So the
> kernel may suport files bigger than 2GB (I really don´t know, I just know
> that in my case with fat32 it did not and I saw this too with some oracle
> databases that could not be used when they grow and reach 2GB, may be a
> library problem too).
>
> That´s all.
> ----- Original Message -----
> From: "Jeff Chua" <jchua@fedex.com>
> To: "Andreas Dilger" <adilger@turbolabs.com>
> Cc: "Marcelo Borges Ribeiro" <marcelo@datacom-telematica.com.br>; "Tyler
> BIRD" <birdty@uvsc.edu>; <linux-kernel@vger.kernel.org>
> Sent: Friday, November 23, 2001 12:35 AM
> Subject: Re: Filesize limit on SMBFS
>
>
> > On Thu, 22 Nov 2001, Andreas Dilger wrote:
> >
> > > VFAT does have a 2GB limit, AFAIK, but I could be wrong.
> >
> > Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.
> >
> > I'm using 3GB for VFAT partition.
> >
> > Jeff.
> >
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> >
>

Re: Filesize limit on SMBFS

[Mike Eldridge]

On Fri, Nov 23, 2001 at 09:10:24AM -0200, Marcelo Borges Ribeiro wrote:
> I have fat32 partition, but the problem isn´t the size of partition it is
> 8GB. The problem is that if you want to
> create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2
> to a reiserfs) to an available fat32 space, in my case the backup size is
> allways 2GB and when I tried to extract back I saw "unexpected end of file".
> So I thought it was that famous kernel limitation of 2GB under any kind of
> partition, but i was informed that fat has this limitation too. So the
> kernel may suport files bigger than 2GB (I really don´t know, I just know
> that in my case with fat32 it did not and I saw this too with some oracle
> databases that could not be used when they grow and reach 2GB, may be a
> library problem too).

ext2 has a 2GB filesize limitation.

-mike

--------------------------------------------------------------------------
   /~\  The ASCII                       all that is gold does not glitter
   \ /  Ribbon Campaign                 not all those who wander are lost
    X   Against HTML                          -- jrr tolkien
   / \  Email!

          radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd           
--------------------------------------------------------------------------

Re: Filesize limit on SMBFS

[war]

To put it simply, ext2 does not have a 2GB filesize limitation anymore, (in newer
distributions).

Mike Eldridge wrote:

> On Fri, Nov 23, 2001 at 09:10:24AM -0200, Marcelo Borges Ribeiro wrote:
> > I have fat32 partition, but the problem isn´t the size of partition it is
> > 8GB. The problem is that if you want to
> > create a cpio backup of a linux system 3.5GB (I did that to reformat a ext2
> > to a reiserfs) to an available fat32 space, in my case the backup size is
> > allways 2GB and when I tried to extract back I saw "unexpected end of file".
> > So I thought it was that famous kernel limitation of 2GB under any kind of
> > partition, but i was informed that fat has this limitation too. So the
> > kernel may suport files bigger than 2GB (I really don´t know, I just know
> > that in my case with fat32 it did not and I saw this too with some oracle
> > databases that could not be used when they grow and reach 2GB, may be a
> > library problem too).
>
> ext2 has a 2GB filesize limitation.
>
> -mike
>
> --------------------------------------------------------------------------
>    /~\  The ASCII                       all that is gold does not glitter
>    \ /  Ribbon Campaign                 not all those who wander are lost
>     X   Against HTML                          -- jrr tolkien
>    / \  Email!
>
>           radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd
> --------------------------------------------------------------------------
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Filesize limit on SMBFS

[Andreas Dilger]

On Nov 23, 2001  17:11 -0600, Mike Eldridge wrote:
> ext2 has a 2GB filesize limitation.

Where do you get that idea from.  I have created files up to 4TB (sparse
ones, of course) without problems.  After that you start hitting bugs in
the VFS and ext2 _code_, but you should be able to have up to 16TB files
on a 4kB block ext2 fs.

Please stop spreading misinformation.  Maybe there is a 2GB limitation
in libc, or your tools, or in 2.2 ext2 _implementation_ (which is
fixed if you apply the LFS patch for ext2), but no such limit in the
design of ext2 itself.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Filesize limit on SMBFS

[Mike Eldridge]

On Fri, Nov 23, 2001 at 04:19:47PM -0700, Andreas Dilger wrote:
> On Nov 23, 2001  17:11 -0600, Mike Eldridge wrote:
> > ext2 has a 2GB filesize limitation.
> 
> Where do you get that idea from.  I have created files up to 4TB (sparse
> ones, of course) without problems.  After that you start hitting bugs in
> the VFS and ext2 _code_, but you should be able to have up to 16TB files
> on a 4kB block ext2 fs.
> 
> Please stop spreading misinformation.  Maybe there is a 2GB limitation
> in libc, or your tools, or in 2.2 ext2 _implementation_ (which is
> fixed if you apply the LFS patch for ext2), but no such limit in the
> design of ext2 itself.

oooh.  i apologize.  i remember reading in several different places that
little tidbit i rattled off.

i stand (majorly) corrected.

-mike

--------------------------------------------------------------------------
   /~\  The ASCII                       all that is gold does not glitter
   \ /  Ribbon Campaign                 not all those who wander are lost
    X   Against HTML                          -- jrr tolkien
   / \  Email!

          radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd           
--------------------------------------------------------------------------

Re: Filesize limit on SMBFS

[Matti Aarnio]

On Fri, Nov 23, 2001 at 05:11:57PM -0600, Mike Eldridge wrote:
> ext2 has a 2GB filesize limitation.

  Mike has already realized this, but still...
I have used ext2 at Linux 1.2 with file sizes exceeding 2GB.
The requirement was 64 bit machine in those days, I had Alpha.
(Yes, that was VERY LONG AGO!)

It is very late (2.3/2.4) development that 32-bit machines can
do 2TB files.  Sparse files can indeed be larger up to 4G times
filesystem block size, but ext2 is limited on one physical
partition, and those are still limited on 1 or 2 terabyte range.
(2G * 512 or 4G * 512.)  (long story why that limit is still
in there, mainly because nobody has had need to rework it.)

Limitations on EXT2 (indeed of all "indirect block indexing"
schemes of SysV FS style) are a bit complicated to calculate.
  B = block size in bytes (512 to 4096 bytes)
  addressable_blocks = (B/4)**3 + (B/4)**2 + ...
  max_offset = B * addressable_blocks

So, say:  max_offset = B**4 / 64 + epsilon

That gives you magnitude. Say with 4k blocks (long story why that
is presently maximum block-size) you can have up to 4 TB file size.
(plus a bit over, see the math.)

The 2.5 series may change the underlying block-device layer so that
it can handle larger block devices than 2TB - the 64 bit machines can
handle them, of course, but 32-bit i386 is a bit limited...


> -mike
>           radiusd+mysql: http://www.cafes.net/~diz/kiss-radiusd           


/Matti Aarnio

Re: Filesize limit on SMBFS

[Eric W. Biederman]

Matti Aarnio <matti.aarnio@zmailer.org> writes:

> The 2.5 series may change the underlying block-device layer so that
> it can handle larger block devices than 2TB - the 64 bit machines can
> handle them, of course, but 32-bit i386 is a bit limited...

Definitely.  Right now the page cache on x86 has a limit (per file) of
2^32 * PAGE_SIZE == 2^32 * 2^12 = 2^44 = 16TB.  And I doubt that will
change.  x86 will be going 64bit in the next 2-3 years, at which point
I don't see it paying to push 32bit code into the larger data sizes.
Especially when we our limit is still 2 orders of magnitude larger
then the largest disk manufactured today.

Now the file size limit on x86 can be increased a little by increasing
the internal PAGE_SIZE for the page cache but that will only give us a
bit or two which really isn't significant.

Eric

Re: Filesize limit on SMBFS

[Alan Cox]

> ext2 has a 2GB filesize limitation.

No it doesn't.

Re: Filesize limit on SMBFS

[Anton Altaparmakov]

At 02:35 23/11/01, Jeff Chua wrote:
>On Thu, 22 Nov 2001, Andreas Dilger wrote:
>
> > VFAT does have a 2GB limit, AFAIK, but I could be wrong.
>
>Use "mkdosfs -F32" or use msdos fdisk,format to get >2GB.
>
>I'm using 3GB for VFAT partition.

You mean you have 1) a single file with size 3GiB on a large VFAT partition 
or 2) the VFAT partition is 3GiB in itself?

1) is what we are talking about being limited to 2GiB.

2) Should indeed work fine under Linux and I don't think anyone is saying 
that this doesn't work.

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: Filesize limit on SMBFS

[Jeff Chua]

On Fri, 23 Nov 2001, Anton Altaparmakov wrote:

> You mean you have 1) a single file with size 3GiB on a large VFAT partition
> or 2) the VFAT partition is 3GiB in itself?

Sorry, 3GB partition. But maximum size per file is only 2GB.

Jeff

Re: Filesize limit on SMBFS

[OGAWA Hirofumi]

Jeff Chua <jchua@fedex.com> writes:

> On Fri, 23 Nov 2001, Anton Altaparmakov wrote:
> 
> > You mean you have 1) a single file with size 3GiB on a large VFAT partition
> > or 2) the VFAT partition is 3GiB in itself?
> 
> Sorry, 3GB partition. But maximum size per file is only 2GB.
> 

FYI,

In the ordinary way, FAT16 is 2GiB per file, and 2GiB per partition.
FAT32 is 4GiB per file. (if sector size is 512B)

However, currently vfat of linux is 2GiB per file.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: Filesize limit on SMBFS

[Albert D. Cahalan]

The ext2 and ext3 limits are the same. The kernel does not have
a 2 GB limit, though I'm sure most users believe it does.

Look at this graph:
http://www.cs.uml.edu/~acahalan/linux/ext2.gif

Pay attention to the note on the lower right.

Filesize limit on SMBFS

[Petr Tite(ra]

Hello,

    is maximum file size on SMBFS really 2GB? I cannot create file 
bigger than that.

Petr Titera
P.Titera@century.cz

Re: Filesize limit on SMBFS

[Urban Widmark]

On Thu, 22 Nov 2001, Petr Tite(ra wrote:

>     is maximum file size on SMBFS really 2GB? I cannot create file 
> bigger than that.

Yes.

I have patches if you want to be my victim^Wtester.

You must be using an NT/2k/XP machine as server, win9x has a 4G limit
(vfat limit?).

Let me know which 2.4 kernel you are using. And if you don't already run a
kernel you compiled yourself, please do that first as you must recompile
to test the patches anyway (smbfs as a module is recommended, then you
should be able to only rebuild the modules part).

/Urban

Re: Filesize limit on SMBFS

[Petr Titera]

Urban Widmark wrote:

> On Thu, 22 Nov 2001, Petr Tite(ra wrote:
> 
> 
>>    is maximum file size on SMBFS really 2GB? I cannot create file 
>>bigger than that.
>>
> 
> Yes.
> 
> I have patches if you want to be my victim^Wtester.


I'd like to.

> 
> You must be using an NT/2k/XP machine as server, win9x has a 4G limit
> (vfat limit?).


It's NT.

> 
> Let me know which 2.4 kernel you are using. And if you don't already run a
> kernel you compiled yourself, please do that first as you must recompile
> to test the patches anyway (smbfs as a module is recommended, then you
> should be able to only rebuild the modules part).


I use 2.4.15-pre7 (compiled myself :)

> 
> /Urban
> 

Petr

Re: Filesize limit on SMBFS

[Urban Widmark]

On Thu, 22 Nov 2001, Petr Titera wrote:

> I'd like to.

http://www.hojdpunkten.ac.se/054/samba/lfs.html

There are some short instructions there. A small change to samba is also
needed, at least with 2.2.2. Although it has been tested, it should not be
trusted. This is experimental code, it may eat your files.

I wrote this some time ago but 2.5 just wouldn't open ... and not enough
testing for 2.4.

/Urban

Re: Filesize limit on SMBFS

[Anton Altaparmakov]

Urban Widmark wrote:
>On Thu, 22 Nov 2001, Petr Tite(ra wrote:
>>is maximum file size on SMBFS really 2GB? I cannot create file bigger 
>>than that.
>Yes.
>I have patches if you want to be my victim^Wtester.

I am in a masochistic mood today so can I be your victim, too? (-;

Seriously, I can test with all of w9x/NT/2k/XP as servers and I need the 
2GB limit aleviated, too, so please email me the [gb]zipped patch (or a URL).

>>Let me know which 2.4 kernel you are using.

At the moment I am using 2.4.15-pre4 + NTFS TNG but I am happy to use any 
2.4.x kernel. make bzImage on my athlon takes only 3 minutes...

Cheers,

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/