fs block size and PAGE_CACHE_SIZE

fs block size and PAGE_CACHE_SIZE

[David Chow]

Hi all,

I've got a question regarding filesystem block size (the arbitary internal fs block size) between the system's PAGE_CACHE_SIZE . Since the most common block size of fs in Linux is 4k, it is because it provides easy implementation of fs which directly matches the PAGE_CACHE_SIZE . As you know, new CPUs such as IA-64's can have a page size of 16k and above, I am wondering is it possible to implement an fs that has 16k block size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system. Firstly, I think of working on a readpage() aop on a non aligned page no (in the middle of a block), how could I efficient populate one 16k block to page cache(4 pages) at one readpage() op? Since the fs driver could not really create a mapped page or telling whether a page (page no) is in the page cache or not. From my knowledge, one way of doing this
  is to use the read_cache_page() callback to tell whether the page is in page cache by knowing wether the callback is triggered. If I simply call read_cache_page() to aligned page in readpage() aop of unaligned pages, even the aligned page is up_to_date afterwards, how would it possible for the readpage() process to return an up_to_date page (guaranteed) for unaligned pages? I thought of locking the page, but the problem is that readpage() is called with the page locked, and call read_cache_page() on a cached aligned page will not trigger an fs read (page cache hit of aligned page doesn't mean an up_to_date for unaligned page). Since my fs implementation would be very inefficient if doing a reading in the middle of a block (compression file systems) or the overhead of reading will be 4 times higher on small page cache size systems. I am pleased to hear any suggestions on such implementations. Thanks.

regards,
David Chow

Re: fs block size and PAGE_CACHE_SIZE

[Szakacsits Szabolcs]

On Wed, 7 May 2003, David Chow wrote:

> size of 16k and above, I am wondering is it possible to implement an fs
> that has 16k block size on a 4k system which this fs can be used both
> on IA-32 and IA-64 systems. In other words, is it possible (or someone
> already done) to implement a file system that has 16k block size on a
> 4k paging system.

The new NTFS driver, in 2.5 or as a patch to 2.4.20, supports all
power of 2 block sizes between 512 bytes and 64kB.

	Szaka

Re: fs block size and PAGE_CACHE_SIZE

[Bryan Henderson]

I don't know why there would be any issue having page size != block size,
as long as one is a multiple of the other.  Maybe you have a particular
issue in mind?

I know one area where having a block larger than a page is a pain:  When
you allocate a new block in order to write just one page, you have to
separately initialize the rest of the block.

>how could I efficient populate one 16k block to page cache(4 pages) at one
readpage() op?

Why would you want to?  If someone wants to access those other 3 pages,
they'll have page faults of their own.

Re: fs block size and PAGE_CACHE_SIZE

[Phillip Lougher]

Bryan Henderson wrote:

>
>>how could I efficient populate one 16k block to page cache(4 pages) at one
>>
>readpage() op?
>
>Why would you want to?  If someone wants to access those other 3 pages,
>they'll have page faults of their own.
>
In an uncompressed filesystem this is not a problem because the 
additional page faults (read_page), will read the pages out of the block 
cache.   However, for a compressed filesystem,   the 16K block would 
have to be (re-)decompressed evey time, because the block is compressed 
in the block cache.  It is much more efficient to push the extra 
decompressed pages into the page cache.

Regards

Phillip Lougher

Re: fs block size and PAGE_CACHE_SIZE

[David Chow]

Bryan Henderson wrote:

>
>
>I don't know why there would be any issue having page size != block size,
>as long as one is a multiple of the other.  Maybe you have a particular
>issue in mind?
>
>I know one area where having a block larger than a page is a pain:  When
>you allocate a new block in order to write just one page, you have to
>separately initialize the rest of the block.
>
>  
>
>>how could I efficient populate one 16k block to page cache(4 pages) at one
>>    
>>
>readpage() op?
>
>Why would you want to?  If someone wants to access those other 3 pages,
>they'll have page faults of their own.
>  
>
Yes, for uncompressed file systems it doesn't really matter. If I am 
writing a file system that supports 16k blocks compression, I have to 
spend 3 extra (12k extra) compression work for only one 4k data. Since 
there is no way for me to read in the middle of the block. If the extra 
12k data read from the block didn't put into page cache and set 
up-to-date, it will be wasted. That's why.

regards,
David Chow

Re: fs block size and PAGE_CACHE_SIZE

[Trond Myklebust]

>>>>> " " == David Chow <davidchow@shaolinmicro.com> writes:

     > In other words, is it possible (or someone already done) to
     > implement a file system that has 16k block size on a 4k paging
     > system.

The NFS client does that. In 2.4.x, I had to invent a custom system
for coalescing the pages, but in 2.5.x, Andrew added the readpages()
op, and the ability to control page readahead via the backing_dev_info
struct.

Cheers,
  Trond

Re: fs block size and PAGE_CACHE_SIZE

[Trond Myklebust]

>>>>> " " == David Chow <davidchow@shaolinmicro.com> writes:

     > How can you make sure there are no race conditions in
     > populating multiple page caches and making them up-to-date
     > afterwards? Since readpage only make sures you lock one page,

Have a browse through mm/readahead.c (in particular read_pages() and
read_cache_pages()).
The page is is anonymous when it gets locked, and is only added to the
page cache) if there is no existing page. The lock is only released by
the filler.

     > If readpages() is added for 2.5, it doesn't mean read page
     > don't have to implement readpage(), right?

Right. You still have to implement readpage.

Cheers,
  Trond

Re: fs block size and PAGE_CACHE_SIZE

[David Chow]

Trond Myklebust wrote:

>>>>>>" " == David Chow <davidchow@shaolinmicro.com> writes:
>>>>>>            
>>>>>>
>
>     > How can you make sure there are no race conditions in
>     > populating multiple page caches and making them up-to-date
>     > afterwards? Since readpage only make sures you lock one page,
>
>Have a browse through mm/readahead.c (in particular read_pages() and
>read_cache_pages()).
>The page is is anonymous when it gets locked, and is only added to the
>page cache) if there is no existing page. The lock is only released by
>the filler.
>
>     > If readpages() is added for 2.5, it doesn't mean read page
>     > don't have to implement readpage(), right?
>
>Right. You still have to implement readpage.
>
So it is impossible to do under 2.4? Since all the design of VFS and 
page cache assumes the underlying fs uses a 4k page or something 
equivalent less to the size of a PAGE_CACHE_SIZE as the largest unit of 
a cache. If you read Phillip Lougher's mail which points to the squash 
fs (read-only), plus the problem of sleeping which really impossible to 
implement a read/write fs on smp systems (which I suppose to, and the 
current fs already supported). If it is impossible, I don't bother to 
think though.

regards,
David Chow

>  
>

Re: fs block size and PAGE_CACHE_SIZE

[Anton Altaparmakov]

On Mon, 12 May 2003, David Chow wrote:
> Trond Myklebust wrote:
> >>>>>>" " == David Chow <davidchow@shaolinmicro.com> writes:
> >     > How can you make sure there are no race conditions in
> >     > populating multiple page caches and making them up-to-date
> >     > afterwards? Since readpage only make sures you lock one page,
> >Have a browse through mm/readahead.c (in particular read_pages() and
> >read_cache_pages()).
> >The page is is anonymous when it gets locked, and is only added to the
> >page cache) if there is no existing page. The lock is only released by
> >the filler.
> >
> >     > If readpages() is added for 2.5, it doesn't mean read page
> >     > don't have to implement readpage(), right?
> >
> >Right. You still have to implement readpage.
> >
> So it is impossible to do under 2.4? Since all the design of VFS and 
> page cache assumes the underlying fs uses a 4k page or something 
> equivalent less to the size of a PAGE_CACHE_SIZE as the largest unit of 
> a cache. If you read Phillip Lougher's mail which points to the squash 
> fs (read-only), plus the problem of sleeping which really impossible to 
> implement a read/write fs on smp systems (which I suppose to, and the 
> current fs already supported). If it is impossible, I don't bother to 
> think though.

The new ntfs driver reads several (compressed data) pages transparently
and at once using readpage and it works just fine both smp and preemption
both on 2.4 and 2.5.  Basically I see it as intelligent readahead.  You
can see the 2.5 code in the vanilla kernel and the 2.4 code by downloading
the 2.4 kernel patch from: 

	http://linux-ntfs.sf.net/downloads.html

Or by pulling from our BK repository:

	bk pull bk://linux-ntfs.bkbits.net/ntfs-2.4

You can also view the BK tree with your webbrowser:

	http://linux-ntfs.bkbits.net:8080/ntfs-2.4/src?nav=index.html

You want to look at the files:

	fs/ntfs/aops.c and fs/ntfs/compress.c.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: fs block size and PAGE_CACHE_SIZE

[Phillip Lougher]

David Chow wrote:

> Trond Myklebust wrote:
>
>>>>>>> " " == David Chow <davidchow@shaolinmicro.com> writes:
>>>>>>>           
>>>>>>
>>
>>     > How can you make sure there are no race conditions in
>>     > populating multiple page caches and making them up-to-date
>>     > afterwards? Since readpage only make sures you lock one page,
>>
>> Have a browse through mm/readahead.c (in particular read_pages() and
>> read_cache_pages()).
>> The page is is anonymous when it gets locked, and is only added to the
>> page cache) if there is no existing page. The lock is only released by
>> the filler.
>>
>>     > If readpages() is added for 2.5, it doesn't mean read page
>>     > don't have to implement readpage(), right?
>>
>> Right. You still have to implement readpage.
>>
> So it is impossible to do under 2.4? Since all the design of VFS and 
> page cache assumes the underlying fs uses a 4k page or something 
> equivalent less to the size of a PAGE_CACHE_SIZE as the largest unit 
> of a cache. If you read Phillip Lougher's mail which points to the 
> squash fs (read-only), plus the problem of sleeping which really 
> impossible to implement a read/write fs on smp systems (which I 
> suppose to, and the current fs already supported). If it is 
> impossible, I don't bother to think though.


Hi,

I think you slightly mis-understood my post.  There is no problem doing 
this with SMP or pre-emptive kernels.

What I did say was to point out it was better to use grab_page_nowait() 
rather than read_cache_page.  The problem with read_cache_page is not to 
with smp or pre-emptive kernels, it is to do with the fact 
read_cache_page tries to grab the lock of the page it is called to fill. 
 If the page is already locked by another process, it will sleep on the 
lock.  Grab_page_nowait() doesn't wait if the page is locked.

Regards

Phillip Lougher

>
> regards,
> David Chow
>
>>  
>>
>
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: fs block size and PAGE_CACHE_SIZE

[David Chow]

>
>
>> So it is impossible to do under 2.4? Since all the design of VFS and 
>> page cache assumes the underlying fs uses a 4k page or something 
>> equivalent less to the size of a PAGE_CACHE_SIZE as the largest unit 
>> of a cache. If you read Phillip Lougher's mail which points to the 
>> squash fs (read-only), plus the problem of sleeping which really 
>> impossible to implement a read/write fs on smp systems (which I 
>> suppose to, and the current fs already supported). If it is 
>> impossible, I don't bother to think though.
>
>
> Hi,
>
> I think you slightly mis-understood my post. There is no problem doing 
> this with SMP or pre-emptive kernels.
>
> What I did say was to point out it was better to use 
> grab_page_nowait() rather than read_cache_page. The problem with 
> read_cache_page is not to with smp or pre-emptive kernels, it is to do 
> with the fact read_cache_page tries to grab the lock of the page it is 
> called to fill. If the page is already locked by another process, it 
> will sleep on the lock. Grab_page_nowait() doesn't wait if the page is 
> locked.
>
> Regards
>
> Phillip Lougher
>
Thanks for your clarifications. But what to do if one page is locked out 
of 4 (a 16k block)? It is easy when readpage() so that I just fill the 
reset 12k with up-to-date data. But for write, it is likely I need 
something to compress, right? If I simply grab the page data without 
considering the lock, corruption will likely to occur. I should sleep 
and wait for the page lock since it might be never unlocked for some 
mmapped and locked pages in different vmarea. Have you consider this at 
all? This is where it comes different with rw fs.

regards,
David Chow

Re: fs block size and PAGE_CACHE_SIZE

[Phillip Lougher]

David Chow wrote:

>Hi all,
>  I am wondering is it possible to implement an fs that has 16k block size on a 4k system which this fs can be used both on IA-32 and IA-64 systems. In other words, is it possible (or someone already done) to implement a file system that has 16k block size on a 4k paging system.
>

Both zisofs and squashfs (http://squashfs.sourceforge.net) use larger 
than 4K blocks.  In the case of squashfs, it can uses blocks up to 32K. 
 Both of these are compressed read-only filesystems.

> Firstly, I think of working on a readpage() aop on a non aligned page no (in the middle of a block), how could I efficient populate one 16k block to page cache(4 pages) at one readpage() op? Since the fs driver could not really create a mapped page or telling whether a page (page no) is in the page cache or not. From my knowledge, one way of doing this is to use the read_cache_page() callback to tell whether the page is in page cache by knowing wether the callback is triggered. If I simply call read_cache_page() to aligned page in readpage() aop of unaligned pages, even the aligned page is up_to_date afterwards, how would it possible for the readpage() process to return an up_to_date page (guaranteed) for unaligned pages? I thought of locking the page, but the problem is that readpage() is called with the page locked, and call read_cache_page() on a cached aligned page will not trigger an fs read (page cache hit of aligned page doesn't mean an up_to_date for unaligned page)
 . Since my fs implementation would be very inefficient if doing a reading in the middle of a block (compression file systems) or the overhead of reading will be 4 times higher on small page cache size systems. I am pleased to hear any suggestions on such implementations. Thanks.
>

Squashfs in the read_page routine pushes the extra pages  (i.e. 3 extra 
4K pages in the case of a 16K block) into the page cache using 
grab_page_nowait(), which returns a page with the appropriate mapping to 
be filled and marked uptodate.  To avoid race conditions this routine 
will not sleep if the page is already locked.

Using read_cache_page() is dangerous, because it sleeps if the page you 
are trying to fill is already locked.  When writing squashfs I hit the 
problem where read_page was being called almost simultaneously for 
different pages which were in the same (32K) block.  The first 
invocation read the 32K block and locked it, then tried to push the 
pages into the page cache using read_cache_page().  However, 
read_cache_page slept when it tried to push the page that the second 
invocation had locked.  The second invocation itself was sleeping 
holding that lock, whilst waiting for the overal 32K block lock .

Regards

Phillip Lougher

>
>
>regards,
>David Chow
>
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: fs block size and PAGE_CACHE_SIZE

[David Chow]

Phillip Lougher wrote:

> David Chow wrote:
>
>> Hi all,
>>  I am wondering is it possible to implement an fs that has 16k block 
>> size on a 4k system which this fs can be used both on IA-32 and IA-64 
>> systems. In other words, is it possible (or someone already done) to 
>> implement a file system that has 16k block size on a 4k paging system.
>>
>
> Both zisofs and squashfs (http://squashfs.sourceforge.net) use larger 
> than 4K blocks.  In the case of squashfs, it can uses blocks up to 
> 32K. Both of these are compressed read-only filesystems.
>
>> Firstly, I think of working on a readpage() aop on a non aligned page 
>> no (in the middle of a block), how could I efficient populate one 16k 
>> block to page cache(4 pages) at one readpage() op? Since the fs 
>> driver could not really create a mapped page or telling whether a 
>> page (page no) is in the page cache or not. From my knowledge, one 
>> way of doing this is to use the read_cache_page() callback to tell 
>> whether the page is in page cache by knowing wether the callback is 
>> triggered. If I simply call read_cache_page() to aligned page in 
>> readpage() aop of unaligned pages, even the aligned page is 
>> up_to_date afterwards, how would it possible for the readpage() 
>> process to return an up_to_date page (guaranteed) for unaligned 
>> pages? I thought of locking the page, but the problem is that 
>> readpage() is called with the page locked, and call read_cache_page() 
>> on a cached aligned page will not trigger an fs read (page cache hit 
>> of aligned page doesn't mean an up_to_date for unaligned page). Since 
>> my fs implementation would be very inefficient if doing a reading in 
>> the middle of a block (compression file systems) or the overhead of 
>> reading will be 4 times higher on small page cache size systems. I am 
>> pleased to hear any suggestions on such implementations. Thanks.
>
>>
>
> Squashfs in the read_page routine pushes the extra pages  (i.e. 3 
> extra 4K pages in the case of a 16K block) into the page cache using 
> grab_page_nowait(), which returns a page with the appropriate mapping 
> to be filled and marked uptodate.  To avoid race conditions this 
> routine will not sleep if the page is already locked.

The case for sleep or no sleep only works on uni processor systems and 
non-preemptive kernel. Since the file system is suppose to work on smp 
systems which sleeping or not doesn't really helps. As you know, cases 
where simultaneous read to the same block really block my head.

>
> Using read_cache_page() is dangerous, because it sleeps if the page 
> you are trying to fill is already locked.  When writing squashfs I hit 
> the problem where read_page was being called almost simultaneously for 
> different pages which were in the same (32K) block.  The first 
> invocation read the 32K block and locked it, then tried to push the 
> pages into the page cache using read_cache_page().  However, 
> read_cache_page slept when it tried to push the page that the second 
> invocation had locked.  The second invocation itself was sleeping 
> holding that lock, whilst waiting for the overal 32K block lock .

Thanks for your pointers.

regards,
David Chow