Re: Swap maximum size documented ?

[Bill Davidsen <davidsen@tmr.com>]

William Lee Irwin III wrote:

>William Lee Irwin III wrote:
>  
>
>>>Without CONFIG_HIGHMEM64G=y you have:
>>>32 swapfiles, max swapfile size of 64GB.
>>>With CONFIG_HIGHMEM64G=y you have:
>>>32 swapfiles, max swapfile size of 512GB.
>>>      
>>>
>
>On Wed, Jun 01, 2005 at 03:10:59PM -0400, Bill Davidsen wrote:
>  
>
>>Does this apply to mmap as well? I have an application which currently 
>>uses 9TB of data, and one thought to boost performance was to mmap the 
>>data. Unfortunately, I know 16TB isn't going to be enough for more than 
>>a few more years :-(
>>    
>>
>
>This only applies to swapping on ia32/i386.
>
>mmap() is limited only by file offsets, which are fully 32-bit on
>32-bit systems. remap_file_pages() is limited by PTE_FILE_MAX_BITS,
>which is fully 32-bit with CONFIG_HIGHMEM64G=y on i386 but only 29 bit
>without it on i386. In general checking for PTE_FILE_MAX_BITS on the
>relevant architecture should answer your question for remap_file_pages(),
>and BITS_PER_LONG for mmap(). The swap limits for other architectures
>will also differ and you generally have to look at the swp_entry/pte
>encoding/decoding macros to decipher what the precise limits are
>(though a quick hacky C program can help discover them for you).
>Generally you get the filesizes by PAGE_SIZE << X_FILE_OFFSET_BITS.
>
>It is in principle possible to sweep the kernel to allow larger file
>offsets on 32-bit systems (pgoff_t is something of a preparation for
>that), but I wouldn't advise trying it without rather strong kernel-fu
>and much willingness to debug it by one's self, and that with a common
>failure mode of fs data corruption. Widening swp_entry_t is slightly
>harder as the ptes have limited capacity so you have to somehow allocate
>extra data in a deadlock-free manner, but one also has less disruptive
>failure modes. I suspect you're not itching to implement these things.
>
>One thing to keep in mind is that these are only permissible filesizes.
>Virtualspace must be managed properly for windowing where it's limited
>and to prevent pagetable proliferation where it's not.
>

Thank you for taking the time to give me such a complete reply (I saved 
it to prevent asking similar in the future). Hopefully I will be able to 
leave the problem to someone else by then.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979