From: Phillip Susi <psusi@cfl.rr.com>
To: Olivier Galibert <galibert@pobox.com>,
"Hack inc." <linux-kernel@vger.kernel.org>
Subject: Re: Reading a bunch of file as fast a possible
Date: Tue, 31 Oct 2006 13:45:44 -0500 [thread overview]
Message-ID: <454799D8.1060908@cfl.rr.com> (raw)
In-Reply-To: <20061031172258.GB8230@dspnet.fr.eu.org>
Yes, it sounds like you are looking for aio, possibly with O_DIRECT if
you would prefer that the reads not churn the cache.
Olivier Galibert wrote:
> After searching for kinda-keywords in a locked-in-memory index, I get
> a list of 50-100 files out of several hundred thousands I want to read
> as fast as possible. I can ensure that the directory structure in hot
> in the dcache by re-reading it from time to time, but there isn't
> enough memory to lock the documents there. So I'd like to read 50-100
> files for which I have the sizes (I put them in the index) and memory
> space as fast as possible (less than 0.1s would be great) from
> cold-ish cache.
>
> The best way is I think to find a way to give all the requests to the
> system and have it sort them optimally at the elevator level. But how
> can I do that? Can aio do it, or something else?
>
> OG.
prev parent reply other threads:[~2006-10-31 18:45 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-10-31 17:22 Reading a bunch of file as fast a possible Olivier Galibert
2006-10-31 18:45 ` Phillip Susi [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=454799D8.1060908@cfl.rr.com \
--to=psusi@cfl.rr.com \
--cc=galibert@pobox.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.