Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Leonardo Chiquitto]

On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB when
>  it was observed that struct stat only had a 32-bit st_blocks field.
>
>  Since then, both the kernel and glibc have evolved as well and now both
>  support 64-bit st_blocks. Applications that can't deal with these ranges
>  are assumed to be "legacy" or "broken." File systems now routinely
>  support file sizes much larger than can be represented by 2^32 * 512.
>
>  But we never revisited that limitation. ReiserFS has always been able to
>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>  limitation has prevented that.
>
>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>  appropriate value. I noticed that XFS adjusts the limit based on the
>  CPU but I'd prefer to err on the side of compatibility and place the
>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>  supported by the file system. At a 4k block size, this is conveniently
>  also the advertised maximum file size of reiserfs.
>
>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>         on 32-bit systems.
>
>  This bug is tracked at: https://bugzilla.novell.com/show_bug.cgi?id=592100
>
> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
> - ---
>  fs/reiserfs/super.c |   17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
>
> - --- a/fs/reiserfs/super.c
> +++ b/fs/reiserfs/super.c
> @@ -1309,6 +1309,18 @@ out_err:
>        return err;
>  }
>  +static inline loff_t
> +reiserfs_max_file_offset(struct super_block *sb)
> +{
> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
> +
> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
> +
> +       return min(fs_max, page_cache_max);
> +}
> +
>  static int read_super_block(struct super_block *s, int offset)
>  {
>        struct buffer_head *bh;
> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>        s->dq_op = &reiserfs_quota_operations;
>  #endif
>  -      /* new format is limited by the 32 bit wide i_blocks field, want to
> - -      ** be one full block below that.
> - -      */
> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>        return 0;
>  }

Hello ReiserFS developers,

Any chance to have this patch submitted to 2.6.35? We're hitting the 2TB
file limit here and this patch resolves the problem.

Thanks,
Leonardo
--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Edward Shishkin]

Leonardo Chiquitto wrote:
> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>   
>>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB when
>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>
>>  Since then, both the kernel and glibc have evolved as well and now both
>>  support 64-bit st_blocks. Applications that can't deal with these ranges
>>  are assumed to be "legacy" or "broken." File systems now routinely
>>  support file sizes much larger than can be represented by 2^32 * 512.
>>
>>  But we never revisited that limitation. ReiserFS has always been able to
>>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>>  limitation has prevented that.
>>
>>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>>  appropriate value. I noticed that XFS adjusts the limit based on the
>>  CPU but I'd prefer to err on the side of compatibility and place the
>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>  supported by the file system. At a 4k block size, this is conveniently
>>  also the advertised maximum file size of reiserfs.
>>
>>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>>         on 32-bit systems.
>>
>>  This bug is tracked at: https://bugzilla.novell.com/show_bug.cgi?id=592100
>>
>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>> - ---
>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>
>> - --- a/fs/reiserfs/super.c
>> +++ b/fs/reiserfs/super.c
>> @@ -1309,6 +1309,18 @@ out_err:
>>        return err;
>>  }
>>  +static inline loff_t
>> +reiserfs_max_file_offset(struct super_block *sb)
>> +{
>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
>> +
>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>> +
>> +       return min(fs_max, page_cache_max);
>> +}
>> +
>>  static int read_super_block(struct super_block *s, int offset)
>>  {
>>        struct buffer_head *bh;
>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>        s->dq_op = &reiserfs_quota_operations;
>>  #endif
>>  -      /* new format is limited by the 32 bit wide i_blocks field, want to
>> - -      ** be one full block below that.
>> - -      */
>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>        return 0;
>>  }
>>     
>
> Hello ReiserFS developers,
>
> Any chance to have this patch submitted to 2.6.35?

I wouldn't rely on this. Reiserfsprogs also should be aware
of the new limits. It's all long..

>  We're hitting the 2TB
> file limit here and this patch resolves the problem.
>
> Thanks,
> Leonardo
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>   

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/19/2010 06:00 PM, Edward Shishkin wrote:
> Leonardo Chiquitto wrote:
>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>  
>>>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>> when
>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>
>>>  Since then, both the kernel and glibc have evolved as well and now both
>>>  support 64-bit st_blocks. Applications that can't deal with these
>>> ranges
>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>  support file sizes much larger than can be represented by 2^32 * 512.
>>>
>>>  But we never revisited that limitation. ReiserFS has always been
>>> able to
>>>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>>>  limitation has prevented that.
>>>
>>>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>>>  appropriate value. I noticed that XFS adjusts the limit based on the
>>>  CPU but I'd prefer to err on the side of compatibility and place the
>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>  supported by the file system. At a 4k block size, this is conveniently
>>>  also the advertised maximum file size of reiserfs.
>>>
>>>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>>>         on 32-bit systems.
>>>
>>>  This bug is tracked at:
>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>
>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>> - ---
>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>
>>> - --- a/fs/reiserfs/super.c
>>> +++ b/fs/reiserfs/super.c
>>> @@ -1309,6 +1309,18 @@ out_err:
>>>        return err;
>>>  }
>>>  +static inline loff_t
>>> +reiserfs_max_file_offset(struct super_block *sb)
>>> +{
>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
>>> +
>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>> +
>>> +       return min(fs_max, page_cache_max);
>>> +}
>>> +
>>>  static int read_super_block(struct super_block *s, int offset)
>>>  {
>>>        struct buffer_head *bh;
>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>        s->dq_op = &reiserfs_quota_operations;
>>>  #endif
>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>> want to
>>> - -      ** be one full block below that.
>>> - -      */
>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>        return 0;
>>>  }
>>>     
>>
>> Hello ReiserFS developers,
>>
>> Any chance to have this patch submitted to 2.6.35?
> 
> I wouldn't rely on this. Reiserfsprogs also should be aware
> of the new limits. It's all long..

I certainly hope not. Things would be broken already. The 2 TB limit is
2048 * 2^32.

- -Jeff

>>  We're hitting the 2TB
>> file limit here and this patch resolves the problem.
>>
>> Thanks,
>> Leonardo
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe
>> reiserfs-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>   
> 


- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/

iEYEARECAAYFAkv1iTwACgkQLPWxlyuTD7I/yQCcCrGCVlu15YW7tBPbXb08SR7U
xc0AmwT5nDgG9T9apDXyQtqlpkx5vwva
=gb6N
-----END PGP SIGNATURE-----

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Tim Shearouse]

On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>> Leonardo Chiquitto wrote:
>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>
>>>>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>>> when
>>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>>
>>>>  Since then, both the kernel and glibc have evolved as well and now both
>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>> ranges
>>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>>  support file sizes much larger than can be represented by 2^32 * 512.
>>>>
>>>>  But we never revisited that limitation. ReiserFS has always been
>>>> able to
>>>>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>>>>  limitation has prevented that.
>>>>
>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>>>>  appropriate value. I noticed that XFS adjusts the limit based on the
>>>>  CPU but I'd prefer to err on the side of compatibility and place the
>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>>  supported by the file system. At a 4k block size, this is conveniently
>>>>  also the advertised maximum file size of reiserfs.
>>>>
>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>>>>         on 32-bit systems.
>>>>
>>>>  This bug is tracked at:
>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>
>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>> - ---
>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>
>>>> - --- a/fs/reiserfs/super.c
>>>> +++ b/fs/reiserfs/super.c
>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>        return err;
>>>>  }
>>>>  +static inline loff_t
>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>> +{
>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
>>>> +
>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>> +
>>>> +       return min(fs_max, page_cache_max);
>>>> +}
>>>> +
>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>  {
>>>>        struct buffer_head *bh;
>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>  #endif
>>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>>> want to
>>>> - -      ** be one full block below that.
>>>> - -      */
>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>        return 0;
>>>>  }
>>>>
>>>
>>> Hello ReiserFS developers,
>>>
>>> Any chance to have this patch submitted to 2.6.35?
>>
>> I wouldn't rely on this. Reiserfsprogs also should be aware
>> of the new limits. It's all long..
>
> I certainly hope not. Things would be broken already. The 2 TB limit is
> 2048 * 2^32.
>
> - -Jeff
>
>>>  We're hitting the 2TB
>>> file limit here and this patch resolves the problem.
>>>
>>> Thanks,
>>> Leonardo
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> reiserfs-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>
>
>
> - --
> Jeff Mahoney
> SUSE Labs
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.15 (GNU/Linux)
> Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkv1iTwACgkQLPWxlyuTD7I/yQCcCrGCVlu15YW7tBPbXb08SR7U
> xc0AmwT5nDgG9T9apDXyQtqlpkx5vwva
> =gb6N
> -----END PGP SIGNATURE-----
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I do not believe this patch will cause problems. As you mention,
ReiserFS was intended to support a max 16TB filesize.

Edward, it looks to me as though reiserfsprogs will be aware of the
new limits, unless I am missing something (it does not have its own
method for accessing the super block somewhere, correct?).
--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Leonardo Chiquitto]

On Fri, May 21, 2010 at 10:17 AM, Tim Shearouse <t.shearouse@gmail.com> wrote:
> On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>>> Leonardo Chiquitto wrote:
>>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>
>>>>>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>>>> when
>>>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>>>
>>>>>  Since then, both the kernel and glibc have evolved as well and now both
>>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>>> ranges
>>>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>>>  support file sizes much larger than can be represented by 2^32 * 512.
>>>>>
>>>>>  But we never revisited that limitation. ReiserFS has always been
>>>>> able to
>>>>>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>>>>>  limitation has prevented that.
>>>>>
>>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>>>>>  appropriate value. I noticed that XFS adjusts the limit based on the
>>>>>  CPU but I'd prefer to err on the side of compatibility and place the
>>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>>>  supported by the file system. At a 4k block size, this is conveniently
>>>>>  also the advertised maximum file size of reiserfs.
>>>>>
>>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>>>>>         on 32-bit systems.
>>>>>
>>>>>  This bug is tracked at:
>>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>>
>>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>>> - ---
>>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>>
>>>>> - --- a/fs/reiserfs/super.c
>>>>> +++ b/fs/reiserfs/super.c
>>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>>        return err;
>>>>>  }
>>>>>  +static inline loff_t
>>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>>> +{
>>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
>>>>> +
>>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>>> +
>>>>> +       return min(fs_max, page_cache_max);
>>>>> +}
>>>>> +
>>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>>  {
>>>>>        struct buffer_head *bh;
>>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>>  #endif
>>>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>>>> want to
>>>>> - -      ** be one full block below that.
>>>>> - -      */
>>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>>        return 0;
>>>>>  }
>>>>>
>>>>
>>>> Hello ReiserFS developers,
>>>>
>>>> Any chance to have this patch submitted to 2.6.35?
>>>
>>> I wouldn't rely on this. Reiserfsprogs also should be aware
>>> of the new limits. It's all long..
>>
>> I certainly hope not. Things would be broken already. The 2 TB limit is
>> 2048 * 2^32.
>>
>> - -Jeff
>>
>>>>  We're hitting the 2TB
>>>> file limit here and this patch resolves the problem.
>>>>
>>>> Thanks,
>>>> Leonardo
>> - --
>> Jeff Mahoney
>> SUSE Labs
>
> I do not believe this patch will cause problems. As you mention,
> ReiserFS was intended to support a max 16TB filesize.
>
> Edward, it looks to me as though reiserfsprogs will be aware of the
> new limits, unless I am missing something (it does not have its own
> method for accessing the super block somewhere, correct?).

Hello,

I apologize for insisting here but it's really important for us to get
this fixed upstream. Please, can someone submit it for 2.6.35 or,
if the patch is not a good idea, give a little insight on what's wrong?

Thanks,
Leonardo
--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Edward Shishkin]

Leonardo Chiquitto wrote:
> On Fri, May 21, 2010 at 10:17 AM, Tim Shearouse <t.shearouse@gmail.com> wrote:
>   
>> On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>     
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>>>       
>>>> Leonardo Chiquitto wrote:
>>>>         
>>>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>>
>>>>>           
>>>>>>  In its early life, reiserfs had an evolving s_max_bytes. It started out
>>>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>>>>> when
>>>>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>>>>
>>>>>>  Since then, both the kernel and glibc have evolved as well and now both
>>>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>>>> ranges
>>>>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>>>>  support file sizes much larger than can be represented by 2^32 * 512.
>>>>>>
>>>>>>  But we never revisited that limitation. ReiserFS has always been
>>>>>> able to
>>>>>>  support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
>>>>>>  limitation has prevented that.
>>>>>>
>>>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a more
>>>>>>  appropriate value. I noticed that XFS adjusts the limit based on the
>>>>>>  CPU but I'd prefer to err on the side of compatibility and place the
>>>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>>>>  supported by the file system. At a 4k block size, this is conveniently
>>>>>>  also the advertised maximum file size of reiserfs.
>>>>>>
>>>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the math works
>>>>>>         on 32-bit systems.
>>>>>>
>>>>>>  This bug is tracked at:
>>>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>>>
>>>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>>>> - ---
>>>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>>>
>>>>>> - --- a/fs/reiserfs/super.c
>>>>>> +++ b/fs/reiserfs/super.c
>>>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>>>        return err;
>>>>>>  }
>>>>>>  +static inline loff_t
>>>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>>>> +{
>>>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
>>>>>> +
>>>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>>>> +
>>>>>> +       return min(fs_max, page_cache_max);
>>>>>> +}
>>>>>> +
>>>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>>>  {
>>>>>>        struct buffer_head *bh;
>>>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>>>  #endif
>>>>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>>>>> want to
>>>>>> - -      ** be one full block below that.
>>>>>> - -      */
>>>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>>>        return 0;
>>>>>>  }
>>>>>>
>>>>>>             
>>>>> Hello ReiserFS developers,
>>>>>
>>>>> Any chance to have this patch submitted to 2.6.35?
>>>>>           
>>>> I wouldn't rely on this. Reiserfsprogs also should be aware
>>>> of the new limits. It's all long..
>>>>         
>>> I certainly hope not. Things would be broken already. The 2 TB limit is
>>> 2048 * 2^32.
>>>
>>> - -Jeff
>>>
>>>       
>>>>>  We're hitting the 2TB
>>>>> file limit here and this patch resolves the problem.
>>>>>
>>>>> Thanks,
>>>>> Leonardo
>>>>>           
>>> - --
>>> Jeff Mahoney
>>> SUSE Labs
>>>       
>> I do not believe this patch will cause problems. As you mention,
>> ReiserFS was intended to support a max 16TB filesize.
>>
>> Edward, it looks to me as though reiserfsprogs will be aware of the
>> new limits, unless I am missing something (it does not have its own
>> method for accessing the super block somewhere, correct?).
>>     
>
> Hello,
>
> I apologize for insisting here but it's really important for us to get
> this fixed upstream. Please, can someone submit it for 2.6.35

I guess nobody will accept this to 2.6.35..
We only can push it to -mm with the following proceeding
from the akpm's side..

>  or,
> if the patch is not a good idea, give a little insight on what's wrong?
>   

Jeff, did you have any chances to run and fsck this on 32 and 64-bit 
machines?

Thanks,
Edward.

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/15/2010 05:00 PM, Edward Shishkin wrote:
> Leonardo Chiquitto wrote:
>> On Fri, May 21, 2010 at 10:17 AM, Tim Shearouse
>> <t.shearouse@gmail.com> wrote:
>>  
>>> On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>    
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>>>>      
>>>>> Leonardo Chiquitto wrote:
>>>>>        
>>>>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>>>
>>>>>>          
>>>>>>>  In its early life, reiserfs had an evolving s_max_bytes. It
>>>>>>> started out
>>>>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>>>>>> when
>>>>>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>>>>>
>>>>>>>  Since then, both the kernel and glibc have evolved as well and
>>>>>>> now both
>>>>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>>>>> ranges
>>>>>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>>>>>  support file sizes much larger than can be represented by 2^32 *
>>>>>>> 512.
>>>>>>>
>>>>>>>  But we never revisited that limitation. ReiserFS has always been
>>>>>>> able to
>>>>>>>  support larger file sizes (up to 16 TiB, in fact), but the
>>>>>>> s_max_bytes
>>>>>>>  limitation has prevented that.
>>>>>>>
>>>>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a
>>>>>>> more
>>>>>>>  appropriate value. I noticed that XFS adjusts the limit based on
>>>>>>> the
>>>>>>>  CPU but I'd prefer to err on the side of compatibility and place
>>>>>>> the
>>>>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>>>>>  supported by the file system. At a 4k block size, this is
>>>>>>> conveniently
>>>>>>>  also the advertised maximum file size of reiserfs.
>>>>>>>
>>>>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the
>>>>>>> math works
>>>>>>>         on 32-bit systems.
>>>>>>>
>>>>>>>  This bug is tracked at:
>>>>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>>>>
>>>>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>>>>> - ---
>>>>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>>>>
>>>>>>> - --- a/fs/reiserfs/super.c
>>>>>>> +++ b/fs/reiserfs/super.c
>>>>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>>>>        return err;
>>>>>>>  }
>>>>>>>  +static inline loff_t
>>>>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>>>>> +{
>>>>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) -
>>>>>>> sb->s_blocksize;
>>>>>>> +
>>>>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>>>>> +
>>>>>>> +       return min(fs_max, page_cache_max);
>>>>>>> +}
>>>>>>> +
>>>>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>>>>  {
>>>>>>>        struct buffer_head *bh;
>>>>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>>>>  #endif
>>>>>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>>>>>> want to
>>>>>>> - -      ** be one full block below that.
>>>>>>> - -      */
>>>>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>>>>        return 0;
>>>>>>>  }
>>>>>>>
>>>>>>>             
>>>>>> Hello ReiserFS developers,
>>>>>>
>>>>>> Any chance to have this patch submitted to 2.6.35?
>>>>>>           
>>>>> I wouldn't rely on this. Reiserfsprogs also should be aware
>>>>> of the new limits. It's all long..
>>>>>         
>>>> I certainly hope not. Things would be broken already. The 2 TB limit is
>>>> 2048 * 2^32.
>>>>
>>>> - -Jeff
>>>>
>>>>      
>>>>>>  We're hitting the 2TB
>>>>>> file limit here and this patch resolves the problem.
>>>>>>
>>>>>> Thanks,
>>>>>> Leonardo
>>>>>>           
>>>> - -- 
>>>> Jeff Mahoney
>>>> SUSE Labs
>>>>       
>>> I do not believe this patch will cause problems. As you mention,
>>> ReiserFS was intended to support a max 16TB filesize.
>>>
>>> Edward, it looks to me as though reiserfsprogs will be aware of the
>>> new limits, unless I am missing something (it does not have its own
>>> method for accessing the super block somewhere, correct?).
>>>     
>>
>> Hello,
>>
>> I apologize for insisting here but it's really important for us to get
>> this fixed upstream. Please, can someone submit it for 2.6.35
> 
> I guess nobody will accept this to 2.6.35..
> We only can push it to -mm with the following proceeding
> from the akpm's side..
> 
>>  or,
>> if the patch is not a good idea, give a little insight on what's wrong?
>>   
> 
> Jeff, did you have any chances to run and fsck this on 32 and 64-bit
> machines?

Revisiting this since we'd really like to include the patch in our
products. I'm doing this testing today.

- -Jeff

- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/

iEYEARECAAYFAkzOx24ACgkQLPWxlyuTD7I7CQCghqjn8J9gV+D4qsAZUS9UJ1NT
Rm0An2PWE1wVfrPpL1uUyF5E3Y7eWJaY
=iDCV
-----END PGP SIGNATURE-----

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Edward Shishkin]

Jeff Mahoney wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 06/15/2010 05:00 PM, Edward Shishkin wrote:
>   
>> Leonardo Chiquitto wrote:
>>     
>>> On Fri, May 21, 2010 at 10:17 AM, Tim Shearouse
>>> <t.shearouse@gmail.com> wrote:
>>>  
>>>       
>>>> On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>    
>>>>         
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA1
>>>>>
>>>>> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>>>>>      
>>>>>           
>>>>>> Leonardo Chiquitto wrote:
>>>>>>        
>>>>>>             
>>>>>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>>>>
>>>>>>>          
>>>>>>>               
>>>>>>>>  In its early life, reiserfs had an evolving s_max_bytes. It
>>>>>>>> started out
>>>>>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB
>>>>>>>> when
>>>>>>>>  it was observed that struct stat only had a 32-bit st_blocks field.
>>>>>>>>
>>>>>>>>  Since then, both the kernel and glibc have evolved as well and
>>>>>>>> now both
>>>>>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>>>>>> ranges
>>>>>>>>  are assumed to be "legacy" or "broken." File systems now routinely
>>>>>>>>  support file sizes much larger than can be represented by 2^32 *
>>>>>>>> 512.
>>>>>>>>
>>>>>>>>  But we never revisited that limitation. ReiserFS has always been
>>>>>>>> able to
>>>>>>>>  support larger file sizes (up to 16 TiB, in fact), but the
>>>>>>>> s_max_bytes
>>>>>>>>  limitation has prevented that.
>>>>>>>>
>>>>>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a
>>>>>>>> more
>>>>>>>>  appropriate value. I noticed that XFS adjusts the limit based on
>>>>>>>> the
>>>>>>>>  CPU but I'd prefer to err on the side of compatibility and place
>>>>>>>> the
>>>>>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
>>>>>>>>  supported by the file system. At a 4k block size, this is
>>>>>>>> conveniently
>>>>>>>>  also the advertised maximum file size of reiserfs.
>>>>>>>>
>>>>>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the
>>>>>>>> math works
>>>>>>>>         on 32-bit systems.
>>>>>>>>
>>>>>>>>  This bug is tracked at:
>>>>>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>>>>>
>>>>>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>>>>>> - ---
>>>>>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>>>>>
>>>>>>>> - --- a/fs/reiserfs/super.c
>>>>>>>> +++ b/fs/reiserfs/super.c
>>>>>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>>>>>        return err;
>>>>>>>>  }
>>>>>>>>  +static inline loff_t
>>>>>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>>>>>> +{
>>>>>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>>>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) -
>>>>>>>> sb->s_blocksize;
>>>>>>>> +
>>>>>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>>>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>>>>>> +
>>>>>>>> +       return min(fs_max, page_cache_max);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>>>>>  {
>>>>>>>>        struct buffer_head *bh;
>>>>>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>>>>>  #endif
>>>>>>>>  -      /* new format is limited by the 32 bit wide i_blocks field,
>>>>>>>> want to
>>>>>>>> - -      ** be one full block below that.
>>>>>>>> - -      */
>>>>>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>>>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>>>>>        return 0;
>>>>>>>>  }
>>>>>>>>
>>>>>>>>             
>>>>>>>>                 
>>>>>>> Hello ReiserFS developers,
>>>>>>>
>>>>>>> Any chance to have this patch submitted to 2.6.35?
>>>>>>>           
>>>>>>>               
>>>>>> I wouldn't rely on this. Reiserfsprogs also should be aware
>>>>>> of the new limits. It's all long..
>>>>>>         
>>>>>>             
>>>>> I certainly hope not. Things would be broken already. The 2 TB limit is
>>>>> 2048 * 2^32.
>>>>>
>>>>> - -Jeff
>>>>>
>>>>>      
>>>>>           
>>>>>>>  We're hitting the 2TB
>>>>>>> file limit here and this patch resolves the problem.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Leonardo
>>>>>>>           
>>>>>>>               
>>>>> - -- 
>>>>> Jeff Mahoney
>>>>> SUSE Labs
>>>>>       
>>>>>           
>>>> I do not believe this patch will cause problems. As you mention,
>>>> ReiserFS was intended to support a max 16TB filesize.
>>>>
>>>> Edward, it looks to me as though reiserfsprogs will be aware of the
>>>> new limits, unless I am missing something (it does not have its own
>>>> method for accessing the super block somewhere, correct?).
>>>>     
>>>>         
>>> Hello,
>>>
>>> I apologize for insisting here but it's really important for us to get
>>> this fixed upstream. Please, can someone submit it for 2.6.35
>>>       
>> I guess nobody will accept this to 2.6.35..
>> We only can push it to -mm with the following proceeding
>> from the akpm's side..
>>
>>     
>>>  or,
>>> if the patch is not a good idea, give a little insight on what's wrong?
>>>   
>>>       
>> Jeff, did you have any chances to run and fsck this on 32 and 64-bit
>> machines?
>>     
>
> Revisiting this since we'd really like to include the patch in our
> products. I'm doing this testing today.
>   

It seems, getting rid of limits became a popular tendency nowadays..

Edward.

Re: [PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/01/2010 12:04 PM, Edward Shishkin wrote:
> Jeff Mahoney wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 06/15/2010 05:00 PM, Edward Shishkin wrote:
>>  
>>> Leonardo Chiquitto wrote:
>>>    
>>>> On Fri, May 21, 2010 at 10:17 AM, Tim Shearouse
>>>> <t.shearouse@gmail.com> wrote:
>>>>  
>>>>      
>>>>> On Thu, May 20, 2010 at 2:10 PM, Jeff Mahoney <jeffm@suse.com> wrote:
>>>>>           
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>> Hash: SHA1
>>>>>>
>>>>>> On 05/19/2010 06:00 PM, Edward Shishkin wrote:
>>>>>>               
>>>>>>> Leonardo Chiquitto wrote:
>>>>>>>                   
>>>>>>>> On Wed, May 19, 2010 at 4:22 PM, Jeff Mahoney <jeffm@suse.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>                       
>>>>>>>>>  In its early life, reiserfs had an evolving s_max_bytes. It
>>>>>>>>> started out
>>>>>>>>>  at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to
>>>>>>>>> 2 TiB
>>>>>>>>> when
>>>>>>>>>  it was observed that struct stat only had a 32-bit st_blocks
>>>>>>>>> field.
>>>>>>>>>
>>>>>>>>>  Since then, both the kernel and glibc have evolved as well and
>>>>>>>>> now both
>>>>>>>>>  support 64-bit st_blocks. Applications that can't deal with these
>>>>>>>>> ranges
>>>>>>>>>  are assumed to be "legacy" or "broken." File systems now
>>>>>>>>> routinely
>>>>>>>>>  support file sizes much larger than can be represented by 2^32 *
>>>>>>>>> 512.
>>>>>>>>>
>>>>>>>>>  But we never revisited that limitation. ReiserFS has always been
>>>>>>>>> able to
>>>>>>>>>  support larger file sizes (up to 16 TiB, in fact), but the
>>>>>>>>> s_max_bytes
>>>>>>>>>  limitation has prevented that.
>>>>>>>>>
>>>>>>>>>  This patch adds a max_file_offset helper to set s_max_bytes to a
>>>>>>>>> more
>>>>>>>>>  appropriate value. I noticed that XFS adjusts the limit based on
>>>>>>>>> the
>>>>>>>>>  CPU but I'd prefer to err on the side of compatibility and place
>>>>>>>>> the
>>>>>>>>>  limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the
>>>>>>>>> maximum
>>>>>>>>>  supported by the file system. At a 4k block size, this is
>>>>>>>>> conveniently
>>>>>>>>>  also the advertised maximum file size of reiserfs.
>>>>>>>>>
>>>>>>>>>  Update: This version properly extends PAGE_SIZE_CACHE so the
>>>>>>>>> math works
>>>>>>>>>         on 32-bit systems.
>>>>>>>>>
>>>>>>>>>  This bug is tracked at:
>>>>>>>>> https://bugzilla.novell.com/show_bug.cgi?id=592100
>>>>>>>>>
>>>>>>>>> Signed-off-by: Jeff Mahoney <jeffm@suse.com>
>>>>>>>>> - ---
>>>>>>>>>  fs/reiserfs/super.c |   17 +++++++++++++----
>>>>>>>>>  1 file changed, 13 insertions(+), 4 deletions(-)
>>>>>>>>>
>>>>>>>>> - --- a/fs/reiserfs/super.c
>>>>>>>>> +++ b/fs/reiserfs/super.c
>>>>>>>>> @@ -1309,6 +1309,18 @@ out_err:
>>>>>>>>>        return err;
>>>>>>>>>  }
>>>>>>>>>  +static inline loff_t
>>>>>>>>> +reiserfs_max_file_offset(struct super_block *sb)
>>>>>>>>> +{
>>>>>>>>> +       /* Limited by stat_data->sd_blocks, 2^32-1 blocks */
>>>>>>>>> +       loff_t fs_max = ((u64)sb->s_blocksize << 32) -
>>>>>>>>> sb->s_blocksize;
>>>>>>>>> +
>>>>>>>>> +       /* Limited by 32-bit MAX_LFS_FILESIZE */
>>>>>>>>> +       loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
>>>>>>>>> +
>>>>>>>>> +       return min(fs_max, page_cache_max);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>>  static int read_super_block(struct super_block *s, int offset)
>>>>>>>>>  {
>>>>>>>>>        struct buffer_head *bh;
>>>>>>>>> @@ -1398,10 +1410,7 @@ static int read_super_block(struct super
>>>>>>>>>        s->dq_op = &reiserfs_quota_operations;
>>>>>>>>>  #endif
>>>>>>>>>  -      /* new format is limited by the 32 bit wide i_blocks
>>>>>>>>> field,
>>>>>>>>> want to
>>>>>>>>> - -      ** be one full block below that.
>>>>>>>>> - -      */
>>>>>>>>> - -     s->s_maxbytes = (512LL << 32) - s->s_blocksize;
>>>>>>>>> +       s->s_maxbytes = reiserfs_max_file_offset(s);
>>>>>>>>>        return 0;
>>>>>>>>>  }
>>>>>>>>>
>>>>>>>>>                             
>>>>>>>> Hello ReiserFS developers,
>>>>>>>>
>>>>>>>> Any chance to have this patch submitted to 2.6.35?
>>>>>>>>                         
>>>>>>> I wouldn't rely on this. Reiserfsprogs also should be aware
>>>>>>> of the new limits. It's all long..
>>>>>>>                     
>>>>>> I certainly hope not. Things would be broken already. The 2 TB
>>>>>> limit is
>>>>>> 2048 * 2^32.
>>>>>>
>>>>>> - -Jeff
>>>>>>
>>>>>>               
>>>>>>>>  We're hitting the 2TB
>>>>>>>> file limit here and this patch resolves the problem.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Leonardo
>>>>>>>>                         
>>>>>> - -- Jeff Mahoney
>>>>>> SUSE Labs
>>>>>>                 
>>>>> I do not believe this patch will cause problems. As you mention,
>>>>> ReiserFS was intended to support a max 16TB filesize.
>>>>>
>>>>> Edward, it looks to me as though reiserfsprogs will be aware of the
>>>>> new limits, unless I am missing something (it does not have its own
>>>>> method for accessing the super block somewhere, correct?).
>>>>>             
>>>> Hello,
>>>>
>>>> I apologize for insisting here but it's really important for us to get
>>>> this fixed upstream. Please, can someone submit it for 2.6.35
>>>>       
>>> I guess nobody will accept this to 2.6.35..
>>> We only can push it to -mm with the following proceeding
>>> from the akpm's side..
>>>
>>>    
>>>>  or,
>>>> if the patch is not a good idea, give a little insight on what's wrong?
>>>>         
>>> Jeff, did you have any chances to run and fsck this on 32 and 64-bit
>>> machines?
>>>     
>>
>> Revisiting this since we'd really like to include the patch in our
>> products. I'm doing this testing today.
>>   
> 
> It seems, getting rid of limits became a popular tendency nowadays..

Ok, I created a file of size 2.1 TB successfully. fsck worked fine with
a tweak to real_space_diff() in the kernel. Unfortunately, sd_blocks
describes 512 byte blocks. It doesn't contain the number of file system
blocks and translate it to 512B blocks on request. So, it wraps and 'du'
claims the file is 2.1 GB instead.

It seems this is another case where I should've proposed a reiserfs 3.7
with compat fields years ago.

- -Jeff

- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/

iEYEARECAAYFAkzQB3MACgkQLPWxlyuTD7KZ2wCeNUNSF5Zbsw7Wll6Juknuw0iC
qBIAnRq4KnbDYo4Jm0PdsHAzL5iL1Urd
=cXdb
-----END PGP SIGNATURE-----

[PATCH RESUBMIT] reiserfs: Remove 2 TB file size limit

[Jeff Mahoney]

 In its early life, reiserfs had an evolving s_max_bytes. It started out
 at 4 GB, then was raised to MAX_LFS_FILESIZE, then dropped to 2 TiB when
 it was observed that struct stat only had a 32-bit st_blocks field.

 Since then, both the kernel and glibc have evolved as well and now both
 support 64-bit st_blocks. Applications that can't deal with these ranges
 are assumed to be "legacy" or "broken." File systems now routinely
 support file sizes much larger than can be represented by 2^32 * 512.

 But we never revisited that limitation. ReiserFS has always been able to
 support larger file sizes (up to 16 TiB, in fact), but the s_max_bytes
 limitation has prevented that.

 This patch adds a max_file_offset helper to set s_max_bytes to a more
 appropriate value. I noticed that XFS adjusts the limit based on the
 CPU but I'd prefer to err on the side of compatibility and place the
 limit at the smaller of the 32-bit MAX_LFS_FILESIZE and the maximum
 supported by the file system. At a 4k block size, this is conveniently
 also the advertised maximum file size of reiserfs.

 Update: This version properly extends PAGE_SIZE_CACHE so the math works
         on 32-bit systems.

 This bug is tracked at: https://bugzilla.novell.com/show_bug.cgi?id=592100

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
---
 fs/reiserfs/super.c |   17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

--- a/fs/reiserfs/super.c
+++ b/fs/reiserfs/super.c
@@ -1309,6 +1309,18 @@ out_err:
 	return err;
 }
 
+static inline loff_t
+reiserfs_max_file_offset(struct super_block *sb)
+{
+	/* Limited by stat_data->sd_blocks, 2^32-1 blocks */
+	loff_t fs_max = ((u64)sb->s_blocksize << 32) - sb->s_blocksize;
+
+	/* Limited by 32-bit MAX_LFS_FILESIZE */
+	loff_t page_cache_max = (((u64)PAGE_CACHE_SIZE << 31)-1);
+
+	return min(fs_max, page_cache_max);
+}
+
 static int read_super_block(struct super_block *s, int offset)
 {
 	struct buffer_head *bh;
@@ -1398,10 +1410,7 @@ static int read_super_block(struct super
 	s->dq_op = &reiserfs_quota_operations;
 #endif
 
-	/* new format is limited by the 32 bit wide i_blocks field, want to
-	 ** be one full block below that.
-	 */
-	s->s_maxbytes = (512LL << 32) - s->s_blocksize;
+	s->s_maxbytes = reiserfs_max_file_offset(s);
 	return 0;
 }
 
-- 
Jeff Mahoney
SUSE Labs