Re: windows tmem

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, May 28, 2013 at 10:17:00AM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, May 28, 2013 at 09:53:23AM +0000, James Harper wrote:
> > > > A less risky implementation would just use tmem as a write-thru cache and
> > > > then just throw out old pages on an LRU basis or something. Or discard the
> > > > pages from tmem on read but write them back to disk. It kind of sucks the
> > > > usefulness out of it though if you can't avoid the writes, and if windows is
> > > > doing some trickery to page out during periods of low io then I'd be
> > > > upsetting that too.
> > > >
> > > 
> > > This sounds a lot less fragile and the saving on reads to the storage backend
> > > could still be significant.
> > 
> > Should be easy enough to test I guess.
> > 
> > > > Anyway I have written a skeleton fs filter so I can monitor what is going on
> > > > in
> > > > better detail when I get a few minutes. Later versions of Windows might
> > > > make use of discard (trim/unmap) which would solve most of the above
> > > > problems.
> > > >
> > > > There do seem to be some (windows equivalent of) page cache operations
> > > > that could be hooked too... or else the api callback naming is leading me
> > > > astray.
> > > >
> > > 
> > > Sounds interesting. Presumably, if you can reliably intercept all IO on a
> > > pagefile then I guess you could use tmem as a write-back cache in front of
> > > doing your own file i/o down the storage stack, as long as you could reliably
> > > flush it out when necessary. E.g. does windows assume anything about the
> > > pagefile content on resume from S3 or S4?
> > > 
> > 
> > I need to look up if FS filter is notified about power state transitions. There may be a FLUSH of some sort that happens at that time. Newer versions of windows have a thing called 'hybrid suspend', where the hibernate file is written out as if windows were about to be hibernated, but it goes to sleep instead of hibernating but if power is lost a resume is still possible. It may be acceptable to say that tmem = no hibernate. Migrate should be easy enough as I have direct control over that and can make tmem be written back out to the pagefile first.
> > 
> > This all assumes that write back is possible too...
> 
> I am not familiar with the Windows APIs, but it sounds like you
> want to use the tmem ephermeal disk cache as an secondary cache
> (which is BTW what Linux does too).
> 
> That is OK the only thing you need to keep in mind that the
> hypervisor might flush said cache out if it decides to do it
> (say a new guest is launched and it needs the memory that
> said cache is using).
> 
> So the tmem_get might tell that it does not have the page anymore.

Oh and I should mention that I would be more than thrilled to try
this out and see how it works.