Flushing the (page?) cache for a block device

Flushing the (page?) cache for a block device

[Sascha Silbe]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

Hello!

I'd like to do a verification step after writing to a USB stick (block 
device level, no filesystem involved). For this to work as intended, I 
need to ensure the reads are not satisfied from blocks still in cache, 
i.e.: flush "the cache" (page cache IIUC) for the block device in 
question (and _only_ this device).
Some research resulted in the following options:

1. Open the target file using O_DIRECT while writing.
      Doesn't feel like the right thing to do; only want to flush the 
cache, not circumvent it. Might severly limit performance as writes are 
synchronous.
      Interface isn't very nice (need to ensure alignment to page size 
in memory).

2. Use posix_fadvise() with POSIX_FADV_DONTNEED.
      Probably works with the current kernel (haven't tested yet), but 
there's absolutely no guarantee.
      Interface much better than O_DIRECT.

3. Flush the entire page cache using procfs.
      Brute force, last resort. Will severly impact performance of my 
tool (as it's going to write to several devices in parallel).


Is there a better method I've not found yet? Nr. 2 sounds like the way 
to go, but I don't like that it could (silently!) stop working with any 
new kernel version.


PS: Not subscribed, so please CC me.

CU Sascha

-- 
http://sascha.silbe.org/
http://www.infra-silbe.de/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: Flushing the (page?) cache for a block device

[Wu Fengguang]

Hi Sascha,

On Sun, Mar 29, 2009 at 02:54:39PM +0200, Sascha Silbe wrote:
> Hello!
>
> I'd like to do a verification step after writing to a USB stick (block  
> device level, no filesystem involved). For this to work as intended, I  
> need to ensure the reads are not satisfied from blocks still in cache,  
> i.e.: flush "the cache" (page cache IIUC) for the block device in  
> question (and _only_ this device).
> Some research resulted in the following options:
>
> 1. Open the target file using O_DIRECT while writing.
>      Doesn't feel like the right thing to do; only want to flush the  
> cache, not circumvent it. Might severly limit performance as writes are  
> synchronous.
>      Interface isn't very nice (need to ensure alignment to page size in 
> memory).
>
> 2. Use posix_fadvise() with POSIX_FADV_DONTNEED.
>      Probably works with the current kernel (haven't tested yet), but  
> there's absolutely no guarantee.
>      Interface much better than O_DIRECT.

FYI, here is a handy fadvise tool:
        http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz

Usage:
        fadvise /some/file 0 0 dontneed

Thanks,
Fengguang

> 3. Flush the entire page cache using procfs.
>      Brute force, last resort. Will severly impact performance of my  
> tool (as it's going to write to several devices in parallel).
>
>
> Is there a better method I've not found yet? Nr. 2 sounds like the way  
> to go, but I don't like that it could (silently!) stop working with any  
> new kernel version.
>
>
> PS: Not subscribed, so please CC me.
>
> CU Sascha
>
> -- 
> http://sascha.silbe.org/
> http://www.infra-silbe.de/

Re: Flushing the (page?) cache for a block device

[Wu Fengguang]

On Mon, Mar 30, 2009 at 03:40:52PM +0800, Wu Fengguang wrote:
> Hi Sascha,
> 
> On Sun, Mar 29, 2009 at 02:54:39PM +0200, Sascha Silbe wrote:
> > Hello!
> >
> > I'd like to do a verification step after writing to a USB stick (block  
> > device level, no filesystem involved). For this to work as intended, I  
> > need to ensure the reads are not satisfied from blocks still in cache,  
> > i.e.: flush "the cache" (page cache IIUC) for the block device in  
> > question (and _only_ this device).
> > Some research resulted in the following options:
> >
> > 1. Open the target file using O_DIRECT while writing.
> >      Doesn't feel like the right thing to do; only want to flush the  
> > cache, not circumvent it. Might severly limit performance as writes are  
> > synchronous.
> >      Interface isn't very nice (need to ensure alignment to page size in 
> > memory).
> >
> > 2. Use posix_fadvise() with POSIX_FADV_DONTNEED.
> >      Probably works with the current kernel (haven't tested yet), but  
> > there's absolutely no guarantee.
> >      Interface much better than O_DIRECT.
> 
> FYI, here is a handy fadvise tool:
>         http://www.zip.com.au/~akpm/linux/patches/stuff/ext3-tools.tar.gz
> 
> Usage:
>         fadvise /some/file 0 0 dontneed

And I'd recommend to make a fsync() call before posix_fadvise() to
makes it work more reliably, because dirty pages are not guaranteed
to be freed by posix_fadvise().

Thanks,
Fengguang