Re: Question: Modifying kernel to handle all I/O requests without page cache

[Dave Chinner <david@fromorbit.com>]

On Thu, Sep 26, 2019 at 06:42:43PM -0700, Jianshen Liu wrote:
> > But if you are trying to create benchmarks for a specific application, if your
> > benchmarks uses DIO or not, will depend on if the application uses DIO or not.
> 
> This is my main question. I want running an application without
> involving page caching effects even when the application does not
> support DIO.

LD_PRELOAD wrapper for the open() syscall. Check that the target is
a file, then add O_DIRECT to the open flags.

Won't help you for mmap() access that will always use the page
cache, though, so things like executables will always use the page
cache regardless of what tricks you try to play.

So, as Carlos has said, what you want to do is largely impossible to
acheive.

> > All I/O requests submitted using direct IO must be aligned. So, if the
> > application does not issue aligned requests, the IO requests will fail.
> 
> Yes, this is one of the difficulties in my problem. The application
> may not issue offset, length, buffer addressed aligned I/O. Thus, I
> cannot blindly convert application I/O to DIO within the kernel.

LD_PRELOAD wrapper to bounce buffer unaligned read/write() requests.

> > I will hit the same point again :) and my question is: Why? :) Will you be using
> > a custom kernel? With this modification? If not, you will not be gathering
> > trustable data anyway.
> 
> I created a loadable module to patch a vanilla kernel using the kernel
> livepatching mechanism.

That's just asking for trouble. I wouldn't trust a kernel that has
been modified in that way as far as I could throw it.

> > If you are trying to measure an application performance on solution X, well,
> > it is pointless to measure direct IO if the application does not use it or
> > vice-versa, so, modifying an application, again, is not what you will want to do
> > for benchmarking, for sure.
> 
> The point is that I'm not trying to measure the performance of an
> application on solution X. I'm trying to generate a
> platform-independent reference unit for the combination of a storage
> device and the application.

Sounds like an exercise that has no practical use to me - the model
will have to be so generic and full of compromises that it won't be
relevant to real world situations....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com