Re: Query to optimize PageCache size in system

Re: Query to optimize PageCache size in system

[Ravish Tayal]

Hi
I am looking for help to reduce the number of cached pages in the
system. I am using 2.6.12 linux kernel on ARM processor.for my
embedded USB mass storage application.

As in the system when I do read/write operatation from the USB disk
attached to my development boards, I observed steep increase in the
(/proc/meminfo)
Buffers:         48 kB           [From 0KB]
Cached:        17184 kB      [From 8100 KB]


I have few queries :

1. MemInuse = (MemTotal - Memfree) , why it is not equal to (Active
Pages + Inactive Pages+mapped pages + slab + page Tables)
2. As per my understanding  slab page numbers should match exactly
with the total page output of /proc/slabinfo is it correct ?
3. Is there any provision in linux kernel through which I can restrict
the new page allocation for disk operation to not to be added in the
page-cache  ?
4. If I open the file with O_DIRECT Flag would page_cache be avoided,
irrespective it is file read or File write operations?
5. Is pdflush parameters dirty_background_ratio and dirty_ratio would
help be in file read operations also to reduce the page cache eneries.
6. As buffer cached enteries increases to only 48K, does it means
mostly page cache are added by filesystem while actual disk operation
are actual very few.

with regards
Ravish Tayal
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Query to optimize PageCache size in system

[Greg KH]

On Wed, May 13, 2009 at 08:35:52PM +0530, Ravish Tayal wrote:
> Hi
> I am looking for help to reduce the number of cached pages in the
> system. I am using 2.6.12 linux kernel on ARM processor.for my
> embedded USB mass storage application.

Hm, 2.6.12 was released almost 4 years ago.  A few hundred thousand
kernel changes have happened since then, and we really can't do much
about such old kernel releases.

If you can still see the same issues in a more modern kernel release, we
will be glad to help out.  Might I suggest 2.6.29?

Best of luck,

greg k-h

Re: Query to optimize PageCache size in system

[Matthew Wilcox]

On Wed, May 13, 2009 at 08:35:52PM +0530, Ravish Tayal wrote:
> I am looking for help to reduce the number of cached pages in the
> system. I am using 2.6.12 linux kernel on ARM processor.for my
> embedded USB mass storage application.

Why do you think the number of cached pages is a problem?  What problem
are you experiencing that you think is caused by having a large number
of cached pages?

> 3. Is there any provision in linux kernel through which I can restrict
> the new page allocation for disk operation to not to be added in the
> page-cache  ?
> 4. If I open the file with O_DIRECT Flag would page_cache be avoided,
> irrespective it is file read or File write operations?

Question 4 appears to be the answer to question 3.  O_DIRECT works for
both reads and writes, but does have restrictions on alignment.

-- 
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: Query to optimize PageCache size in system

[Jamie Lokier]

Matthew Wilcox wrote:
> On Wed, May 13, 2009 at 08:35:52PM +0530, Ravish Tayal wrote:
> > I am looking for help to reduce the number of cached pages in the
> > system. I am using 2.6.12 linux kernel on ARM processor.for my
> > embedded USB mass storage application.
> 
> Why do you think the number of cached pages is a problem?  What problem
> are you experiencing that you think is caused by having a large number
> of cached pages?

Ravish didn't say if this applies, but cached pages are a problem on
NOMMU platforms because they cause severe placement-fragmentation of
other memory allocations including userspace allocations, and since
those all must be physically contiguous, it's a problem.

> > 3. Is there any provision in linux kernel through which I can restrict
> > the new page allocation for disk operation to not to be added in the
> > page-cache  ?
> > 4. If I open the file with O_DIRECT Flag would page_cache be avoided,
> > irrespective it is file read or File write operations?
> 
> Question 4 appears to be the answer to question 3.  O_DIRECT works for
> both reads and writes, but does have restrictions on alignment.

In theory there are some fadvise calls as an alternative, but I don't
know if they work.

There was also a patch posted a few months ago to limit the amount of
memory used for file page cache to a fraction of the total memory.
That might work.

A better solution to avoid fragmentation is crudely segregating file
cache pages from other allocations.  That's roughly what 2.4 NOMMU
kernels do, but 2.6 doesn't have the option.  (It has other problems
anyway).

-- Jamie

Fwd: Query to optimize PageCache size in system

[Ravish Tayal]

Including Linux mailing list in the mail thread.

Thanks Matthew,
Please see my inline explanations.

On Wed, May 13, 2009 at 9:24 PM, Matthew Wilcox <matthew@wil.cx> wrote:
> On Wed, May 13, 2009 at 08:35:52PM +0530, Ravish Tayal wrote:
>> I am looking for help to reduce the number of cached pages in the
>> system. I am using 2.6.12 linux kernel on ARM processor.for my
>> embedded USB mass storage application.
>

> Why do you think the number of cached pages is a problem?  What problem
> are you experiencing that you think is caused by having a large number
> of cached pages?
>
RT: My intention is to keep the Zone->nr_inactive to be minimum. By
which I means in case File system read operation pages which are in
active list should alow to be part of the page cache, while once moved
to inactive list it should be soon aged out to free list.
Which for embedded scenario would allow low memory devices to keep
good amount of free memory.

Currently I am periodically calling try_to_free_pages to get it done ;-)

>> 3. Is there any provision in linux kernel through which I can restrict
>> the new page allocation for disk operation to not to be added in the
>> page-cache  ?

>> 4. If I open the file with O_DIRECT Flag would page_cache be avoided,
>> irrespective it is file read or File write operations?
>
> Question 4 appears to be the answer to question 3.  O_DIRECT works for
> both reads and writes, but does have restrictions on alignment.
>

RT: I tried to use the O_DIRECT options with bftpd ftp  server +
2.6.12 kernel, In File write operation I have observed no difference
in term page caching behaviour.
Though in that setup file read operation starts failining which I
didn't debug much.

Thanks for your suggestion.
regards
Ravish Tayal
> --
> Matthew Wilcox                          Intel Open Source Technology Centre
> "Bill, look, we understand that you're interested in selling us this
> operating system, but compare it to ours.  We can't possibly take such
> a retrograde step."
>
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html