Flushing block-level read cache for a single (shared) device.

[Vivien Bernet-Rollande <vivien.bernet-rollande@nexen.alterway.fr>]

Hi list.

I have two Xen hypervisors. Virtual machines use raw ISCSI devices for 
storage.

I ran into a cache coherency issue with live migration.

Let's say VM "a" runs on hypervisor "x1", and I want to move it to "x2". 
This works fine.
Now, I let "a" do some file system operations on "x2".

The problem arises when I move "a" back to "x1". What happens is x1 
still has it's old caches for the iscsi LUN. After a few minutes, the 
filesystem of the VM gets heavily corrupted, and is basically unrecoverable.

I'm currently working around this issue by running something like "sync; 
echo 3 > /proc/sys/vm/drop_caches" on the destination before any 
migration. This works, and I no longer have any data corruption. However 
I feel dropping all caches is not a satisfying solution.

I'm looking for a way to invalidate the cache for a single block device. 
I know of BLKFLSBUF to flush write buffers, but I couldn't find anything 
related to read caches.

-- 
Vivien Bernet-Rollande
Systems&  Networking Engineer
Alter Way Hosting