32/64 bit b_rsector type

32/64 bit b_rsector type

[David Chow]

Dear all,

On a native 32-bit Linux system, the bh->b_resetor (type unsigned long) 
max is limited to 16TB. For 64 bit systems, I know it can get through 
this point, but what about if I chanage the bh->b_rsector in fs.h to u64 
(unsigned long long on 32-bit systems)? Can I simply break through the 
16TB on 32-bit Linux by doing so? Please advice with possiblility of 
impacts. Thanks.

regards,
David Chow

Re: 32/64 bit b_rsector type

[Matthew Wilcox]

On Wed, Mar 24, 2004 at 02:36:29PM +0800, David Chow wrote:
> On a native 32-bit Linux system, the bh->b_resetor (type unsigned long) 
> max is limited to 16TB. For 64 bit systems, I know it can get through 
> this point, but what about if I chanage the bh->b_rsector in fs.h to u64 
> (unsigned long long on 32-bit systems)? Can I simply break through the 
> 16TB on 32-bit Linux by doing so? Please advice with possiblility of 
> impacts. Thanks.

The 16TB limit on a block device is due to the page cache -- 4k pages *
4GB of 'unsigned long index'.  Linus has drawn a line in the sand on
that one -- index is not going to be u64.  The only other ways to go
bigger than this are:

 - Use a 64-bit machine (duh ;-)
 - Stop using the page cache for blockdevs (not popular)
 - Steal some bits from elsewhere to increase the size

The last one is probably best exemplified by pgcl.  That increases the
page size so you can have, say, 16k * 4GB which would get you to a 64GB
blockdev limit, or 64k * 4GB limiting you to 256GB.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: 32/64 bit b_rsector type

[Bryan Henderson]

>what about if I chanage the bh->b_rsector in fs.h to u64 
>> (unsigned long long on 32-bit systems)? Can I simply break through the 
>> 16TB on 32-bit Linux by doing so? Please advice with possiblility of 
>> impacts. Thanks.
>
>The 16TB limit on a block device is due to the page cache -- 4k pages *
>4GB of 'unsigned long index'.
>...
>The only other ways to go bigger than this are:
> ...
> - Stop using the page cache for blockdevs (not popular)
>...

Why would we have to stop?  Doesn't this just mean access through the page 
cache would remain limited to 16TB?  What if I don't really care about 
access to the device with read() -- I just want to put a filesystem on it? 
 Would the simple change the bh->b_rsector enable that?

Re: 32/64 bit b_rsector type

[Dave Kleikamp]

On Wed, 2004-03-24 at 11:06, Bryan Henderson wrote:
> 
> Why would we have to stop?  Doesn't this just mean access through the page 
> cache would remain limited to 16TB?  What if I don't really care about 
> access to the device with read() -- I just want to put a filesystem on it? 
>  Would the simple change the bh->b_rsector enable that?

mkfs and fsck may have an issue with the inability to do reads and
writes to the device.
-- 
David Kleikamp
IBM Linux Technology Center

Re: 32/64 bit b_rsector type

[David Chow]

I think page cache will probably limited by the hashing functions which
is type sensitive to page->index.

Dave Kleikamp 提到:

>On Wed, 2004-03-24 at 11:06, Bryan Henderson wrote:
>  
>
>>Why would we have to stop?  Doesn't this just mean access through the page 
>>cache would remain limited to 16TB?  What if I don't really care about 
>>access to the device with read() -- I just want to put a filesystem on it? 
>> Would the simple change the bh->b_rsector enable that?
>>    
>>
For me, I would like to just to put a specialized LVM or 64-bit nbd
server on it.

>mkfs and fsck may have an issue with the inability to do reads and
>writes to the device.
>
For my issue (which I originally brought up) is not targeted to use with
fsck or mkfs . I am researching to build a cluster IP-based storage
infrastructure with special interconnects. We will use software-based
RAID drivers which I can't see direct benefits from using 64-bit
machines. For cost reasons, I would really want to stick with 32-bit
machines wiht high speed CPUs but with the ability to access blocks
beyond 16TB.

Large pages is not my direction, because pages will be send/receive from
multiple IP NIC's which is not a good idea to break things up nor to
send large chunks.

For page cache or buffer cache or any cache, page->index and
bh->b_rsector will definitely have to change to u64 to fullfil the
requirement. And of course, fsck and mkfs will break if this is true, fs
drivers will also have to be patched and checked for type corrections.
Still, I would like to hear more voices from experienced fs developers
(like you guys) to explore potential issues before I actually modify the
code.

Thanks for pointing out. ^_^

regards,
David Chow

>  
>

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html