Re: smart cache. ist is possible?

["Pádraig Brady" <P@draigBrady.com>]

Alexey Fisher wrote:
> Hallo all.
> I found for my self how great is cache in linux. If read one file from
> disk, so i don't need to do it second time, chace will do the job. It
> speed up thing greatly. But i found it not working with realy big files.
> Like i have 4GB RAM, so if i read a file like 4.6GB, cache won't work.
> Is it possible to have some sort of smart cache wich will read for
> exaplme 1GB from disk and other part from cache?
> 
> here is some simple test:
> =====================cache not working==========================
> dd if=dvd.iso of=/dev/null
> 9017680+0 Datensätze ein
> 9017680+0 Datensätze aus
> 4617052160 Bytes (4,6 GB) kopiert, 90,7817 s, *50,9 MB/s*
> 
> dd if=dvd.iso of=/dev/null
> 9017680+0 Datensätze ein
> 9017680+0 Datensätze aus
> 4617052160 Bytes (4,6 GB) kopiert, 90,7817 s, *50,9 MB/s*
> ===============================================================

Right. The cache is being cycled.
I.E. the block you want is never in the cache
as it has previously clobbered be data your reading in.
That's just a consequence of preferring to cache blocks
you have recently read from file, over older blocks.
This is usually the right thing to do, but exactly
the wrong thing to do in your case.

So you would need to provide more info to the
kernel for it to behave as you want.
I.E. never evict a block belonging to the same file
as the block you're trying to insert.

I wonder should posix_fadvise(...POSIX_FADV_SEQUENTIAL)
do what you want. I don't think it does at present.
Note dd doesn't use posix_fadvise() yet, and it probably
should (at least for POSIX_FADV_DONTNEED).
Pity there is no interface like Robert Love's old O_STREAM
patch to just specify the intent for an fd rather than
worrying about ranges.

cheers,
Pádraig.