IDE and locking

IDE and locking

[P]

Hi,

I'm working on an embedded system here that
basically counts ethernet packets. I'm using
2.4.20 and I'm noticing that when files are
read from the compact flash (ext2), ethernet
packets are being dropped.

So to work around the problem, I'm precaching
all the files (12MB) at boot like:
find / -type f | while read file; do dd if=$file of=/dev/null; done

So questions.

1. Is this due to the BKL around the IDE subsystem?
2. Are writes as susceptible as reads to the problem?
    I'm guessing if the compact flash had a write cache
    buffer I would be OK as I only write max 2MB at a time?
3. There was talk about removing the BKL in 2.6 with
    reference to:
    http://sourceforge.net/project/showfiles.php?group_id=8875
    Has this work been included?
4. Is there a max number of files that can be cached by linux?
5. Will the files be removed at any stage from the cache
    if there is no memory pressure?
6. Can I reserve memory for the file cache?

cheers.
-- 
Pádraig Brady - http://www.pixelbeat.org

Re: IDE and locking

[Paul Jackson]

This doesn't answer your real question, but I can't resist
noticing that the above would be more efficient as:

  find / -type f | xargs wc >/dev/null

Several times fewer system calls, with just a few exec's of 'wc',
instead of exec'ing a 'dd' per file, and with just one read per 16 Kb,
instead of both a read and write per 512 bytes.

Granted - I'm being silly to mention this - your compact flash
device is obviously the bottleneck here.  Doesn't really matter
if the cpu spends 1% or 2% of its time outside the idle loop.


> 4. Is there a max number of files that can be cached by linux?

As long as there is no memory pressure, I suspect it keeps caching
more.

> 5. Will the files be removed at any stage from the cache
>     if there is no memory pressure?

I don't think so - stuff seems to stay in cache 'forever',
if no one else wants the memory.

> 6. Can I reserve memory for the file cache?

Not that I know of.  If it were read-only data, I might try a ram disk,
but your application apparently is read-write.

It might be (long shot, here) that just caching the directories and
inodes was enough, without caching the file contents:

  find / ! -true


-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373