Wrapper fs Code (wrapfs)

Wrapper fs Code (wrapfs)

[UZAIR LAKHANI]

Hello All, 

I have some questions regarding the wrapper fs code
downloaded from filesystems.org

The questions are below:

(1) Finding out the meanings of FIST_FILTER_SCA and   
              FIST_FILTER_DATA

(2) How wrapfs handles the dentry and inode cache of
wrapfs and not the actual fs. While reading the code
of struct dentry_operations of wrapfs, I found out
that the operations involved don't do anything to the
wrapfs dentry cache, these operations only deal with
the actual fs dcache using actual fs dentry
operations. 

(3) When I update some file in the actual fs e.g. a
file in the folder on ext3 file system, the changes
done there are not reflected on the same file on
wrapper fs but the vice versa is working correctly.

I want some details regarding these questions. Hope I
am able to convey my point.

Thanks,
Uzair Lakhani,
Karachi, Pakistan.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Wrapper fs Code (wrapfs)

[Josef Sipek]

On Thu, Aug 03, 2006 at 11:09:10PM -0700, UZAIR LAKHANI wrote:
...
> (1) Finding out the meanings of FIST_FILTER_SCA and   

The stackable filesystem changes the size of the data: for example, gzipfs
takes n bytes and compresses them to m bytes.

>               FIST_FILTER_DATA

Data gets transformed, but the size is the same: for example, rot13fs takes
n bytes and runs them though rot13, the result is still n bytes.

> (3) When I update some file in the actual fs e.g. a
> file in the folder on ext3 file system, the changes
> done there are not reflected on the same file on
> wrapper fs but the vice versa is working correctly.

That is an artifact of how the Linux VFS works. When you modify a file
though wrapfs (or any other stackable file system) the wrapfs code gets
called. It then calls the lower file system code on your behalf. When you
modify the lower file directly, ext3 gets the called, but since Linux has no
way for this change to be announced up the stack, wrapfs happily uses the
old, cached data. If you have ideas about how to solve this, please share
them - right now, there is a lot of work being done to get Unionfs (ours)
and eCryptfs into the vanilla linux kernel.

Jeff.

-- 
Linux, n.:
  Generous programmers from around the world all join forces to help you
  shoot yourself in the foot for free. 

Re: Wrapper fs Code (wrapfs)

[UZAIR LAKHANI]

--- Josef Sipek <jsipek@fsl.cs.sunysb.edu> wrote:

> On Thu, Aug 03, 2006 at 11:09:10PM -0700, UZAIR
> LAKHANI wrote:
> ...
> > (1) Finding out the meanings of FIST_FILTER_SCA
> and   
> 
> The stackable filesystem changes the size of the
> data: for example, gzipfs
> takes n bytes and compresses them to m bytes.
> 
> >               FIST_FILTER_DATA
> 
> Data gets transformed, but the size is the same: for
> example, rot13fs takes
> n bytes and runs them though rot13, the result is
> still n bytes.
> 
> > (3) When I update some file in the actual fs e.g.
> a
> > file in the folder on ext3 file system, the
> changes
> > done there are not reflected on the same file on
> > wrapper fs but the vice versa is working
> correctly.
> 
> That is an artifact of how the Linux VFS works. When
> you modify a file
> though wrapfs (or any other stackable file system)
> the wrapfs code gets
> called. It then calls the lower file system code on
> your behalf. When you
> modify the lower file directly, ext3 gets the
> called, but since Linux has no
> way for this change to be announced up the stack,
> wrapfs happily uses the
> old, cached data. If you have ideas about how to
> solve this, please share
> them - right now, there is a lot of work being done
> to get Unionfs (ours)
> and eCryptfs into the vanilla linux kernel.
> 

Hello All,
Thanks for the reply. Can you please tell me where VFS
caches the data, I have an idea about the dentry and
inode cache but how VFS caches the actual data.

Can you please give some details.

Thanks,
Uzair Lakhani,
Karachi, Pakistan.
> Jeff.
> 
> -- 
> Linux, n.:
>   Generous programmers from around the world all
> join forces to help you
>   shoot yourself in the foot for free. 
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Wrapper fs Code (wrapfs)

[Josef Sipek]

On Sun, Aug 06, 2006 at 09:50:53PM -0700, UZAIR LAKHANI wrote:
> Hello All,
> Thanks for the reply. Can you please tell me where VFS
> caches the data, I have an idea about the dentry and
> inode cache but how VFS caches the actual data.
 
The data is cached in the page cache.

Josef "Jeff" Sipek.

-- 
Bad pun of the week: The formula 1 control computer suffered from a race condition

Re: Wrapper fs Code (wrapfs)

[UZAIR LAKHANI]

--- Josef Sipek <jsipek@fsl.cs.sunysb.edu> wrote:

> On Sun, Aug 06, 2006 at 09:50:53PM -0700, UZAIR
> LAKHANI wrote:
> > Hello All,
> > Thanks for the reply. Can you please tell me where
> VFS
> > caches the data, I have an idea about the dentry
> and
> > inode cache but how VFS caches the actual data.
>  
> The data is cached in the page cache.

Hello All,

Can we not by some way invalidate the old page and
force the kernel to read the actual page i.e. can we
not by any way by-pass the page cache for wrapfs read.

How ext3 e.g. always get the updated data on read. Can
anyone give some detail explanations.

Thanks,
Uzair Lakhani,
Karachi, Pakistan.
> 
> Josef "Jeff" Sipek.
> 
> -- 
> Bad pun of the week: The formula 1 control computer
> suffered from a race condition
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Wrapper fs Code (wrapfs)

[Josef Sipek]

On Tue, Aug 08, 2006 at 12:57:29AM -0700, UZAIR LAKHANI wrote:
> --- Josef Sipek <jsipek@fsl.cs.sunysb.edu> wrote:
> > The data is cached in the page cache.
> 
> Can we not by some way invalidate the old page and
> force the kernel to read the actual page i.e. can we
> not by any way by-pass the page cache for wrapfs read.
 
Currently, no. We (the stackable fs) would need to invalidate the page
associated with ext3. ext3 won't invalidate our pages, so we would have to
do it ourselves. To do that, we'd need some kind of callback so that when an
ext3 page changes, we can propagate the change.
 
> How ext3 e.g. always get the updated data on read.

I am not sure what you mean.

Josef "Jeff" Sipek.

-- 
The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all progress
depends on the unreasonable man.
		- George Bernard Shaw

Re: Wrapper fs Code (wrapfs)

[UZAIR LAKHANI]

--- Josef Sipek <jsipek@fsl.cs.sunysb.edu> wrote:

> On Tue, Aug 08, 2006 at 12:57:29AM -0700, UZAIR
> LAKHANI wrote:
> > --- Josef Sipek <jsipek@fsl.cs.sunysb.edu> wrote:
> > > The data is cached in the page cache.
> > 
> > Can we not by some way invalidate the old page and
> > force the kernel to read the actual page i.e. can
> we
> > not by any way by-pass the page cache for wrapfs
> read.
>  
> Currently, no. We (the stackable fs) would need to
> invalidate the page
> associated with ext3. ext3 won't invalidate our
> pages, so we would have to
> do it ourselves. To do that, we'd need some kind of
> callback so that when an
> ext3 page changes, we can propagate the change.

Thanks for the consistent and regular replies. Can you
please give some more details about the callback idea.
Do we will need to change the kernel code to implement
this callback idea but this will require kernel
re-compilation.

Can we solve the problem by implementing our own
address space operations.

One more question, as far as I know, distributed file
systems (e.g NFS) are also like a wrapper fs on server
side. How then these distributed file systems handle
cache coherency problem.

Thanks,
Uzair Lakhani
>  
> > How ext3 e.g. always get the updated data on read.
> 
> I am not sure what you mean.
> 
> Josef "Jeff" Sipek.
> 
> -- 
> The reasonable man adapts himself to the world; the
> unreasonable one
> persists in trying to adapt the world to himself.
> Therefore all progress
> depends on the unreasonable man.
> 		- George Bernard Shaw
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: Wrapper fs Code (wrapfs)

[John T. Kohl]

>>>>> "Josef" == Josef Sipek <jsipek@fsl.cs.sunysb.edu> writes:

Josef> That is an artifact of how the Linux VFS works. When you modify a
Josef> file though wrapfs (or any other stackable file system) the
Josef> wrapfs code gets called. It then calls the lower file system code
Josef> on your behalf. When you modify the lower file directly, ext3
Josef> gets the called, but since Linux has no way for this change to be
Josef> announced up the stack, wrapfs happily uses the old, cached
Josef> data. If you have ideas about how to solve this, please share
Josef> them - right now, there is a lot of work being done to get
Josef> Unionfs (ours) and eCryptfs into the vanilla linux kernel.

While constructed for a vnode system, the ideas in a set of
Ficus-related papers and theses could be helpful here.  This is John
Heidemann's PhD thesis on the topic (covers necessary VM designs,
callbacks, etc.):
           http://www.isi.edu/people/johnh/PAPERS/Heidemann95e.html

-- 
John Kohl
Senior Software Engineer - Rational Software - IBM Software Group
Lexington, Massachusetts, USA
jtk@us.ibm.com
<http://www.ibm.com/software/rational/>

Re: Wrapper fs Code (wrapfs)

[Josef Sipek]

On Mon, Aug 07, 2006 at 04:35:23PM -0400, John T. Kohl wrote:
> >>>>> "Josef" == Josef Sipek <jsipek@fsl.cs.sunysb.edu> writes:
> 
> Josef> That is an artifact of how the Linux VFS works. When you modify a
> Josef> file though wrapfs (or any other stackable file system) the
> Josef> wrapfs code gets called. It then calls the lower file system code
> Josef> on your behalf. When you modify the lower file directly, ext3
> Josef> gets the called, but since Linux has no way for this change to be
> Josef> announced up the stack, wrapfs happily uses the old, cached
> Josef> data. If you have ideas about how to solve this, please share
> Josef> them - right now, there is a lot of work being done to get
> Josef> Unionfs (ours) and eCryptfs into the vanilla linux kernel.
> 
> While constructed for a vnode system, the ideas in a set of
> Ficus-related papers and theses could be helpful here.  This is John
> Heidemann's PhD thesis on the topic (covers necessary VM designs,
> callbacks, etc.):
>            http://www.isi.edu/people/johnh/PAPERS/Heidemann95e.html

I have read the SOSP '95 paper by the same author a while back (I skimmed
thought the paper you linked to, and it looks like chapters 6-8 cover the
SOSP paper material) and I'd like to point out that such architecture is not
trivial to implement - at least as described by the paper. There are,
however, few interesting ideas there that might be worth looking into.

I think that the current thinking is, let's get ecyptfs/unionfs out there,
and then let's fix what _needs_ to be fixed, instead of redoing the page
cache from the ground up. Sure, we might end up with something similar to
what the paper describes, but "Linux is evolution, not intelligent design."

Thanks for the link,

Josef "Jeff" Sipek.

-- 
Bus Error: passangers dumped.