Mirror a file system on the fly

Mirror a file system on the fly

[Dave Schwartz]

Hi list,
 
Not too sure if this is the right forum to ask this question but since
my requirement is around linux filesystems, I shall take this liberty
to post my question.

My requirement is to develop a kernel/user space module to add an
extension to the shell program environment such that this shell forks
a mirror look-alike filesystem of the underlying OS to the programs
run in that particular shell.

 
Was trying to look thru the FAQ and a few list archives to look for
ideas around my requirement. The archives were overwhelming.
 

Any ideas/pointers will be a great help,
Gracias,
decebel

Re: Mirror a file system on the fly

[Dave Kleikamp]

On Fri, 2005-08-19 at 01:10 +0530, Dave Schwartz wrote:
> Hi list,
>  
> Not too sure if this is the right forum to ask this question but since
> my requirement is around linux filesystems, I shall take this liberty
> to post my question.
> 
> My requirement is to develop a kernel/user space module to add an
> extension to the shell program environment such that this shell forks
> a mirror look-alike filesystem of the underlying OS to the programs
> run in that particular shell.
> 
>  
> Was trying to look thru the FAQ and a few list archives to look for
> ideas around my requirement. The archives were overwhelming.
>  
> 
> Any ideas/pointers will be a great help,

The chroot command and mount --bind should give you a good start.

> Gracias,
> decebel
-- 
David Kleikamp
IBM Linux Technology Center

Re: Mirror a file system on the fly

[Ram Pai]

On Thu, 2005-08-18 at 12:40, Dave Schwartz wrote:
> Hi list,
>  
> Not too sure if this is the right forum to ask this question but since
> my requirement is around linux filesystems, I shall take this liberty
> to post my question.
> 
> My requirement is to develop a kernel/user space module to add an
> extension to the shell program environment such that this shell forks
> a mirror look-alike filesystem of the underlying OS to the programs
> run in that particular shell.

u seem to be talking about namespaces, if I get you right.

there is a flag CLONE_NEWNS to the system call 'clone' which does what
u r talking about.

RP




> 
>  
> Was trying to look thru the FAQ and a few list archives to look for
> ideas around my requirement. The archives were overwhelming.
>  
> 
> Any ideas/pointers will be a great help,
> Gracias,
> decebel
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mirror a file system on the fly

[Dave Schwartz]

Hi Ram,
Thanks for the inputs. I was going over the man pages describing the
clone system call and its option of CLONE_NEWNS. Could understand the
description only in parts.

The man page suggests that this flag when set, the cloned child is
started in a new name space, initialized with a copy of the parent.
Now does that mean, a program like a shell when cloned with
CLONE_NEWNS set, will have a copy of file hierarchy of the underlying
parent process?

Gracias,
decebel

  

On 8/19/05, Ram Pai <linuxram@us.ibm.com> wrote:
> On Thu, 2005-08-18 at 12:40, Dave Schwartz wrote:
> > Hi list,
> >
> > Not too sure if this is the right forum to ask this question but since
> > my requirement is around linux filesystems, I shall take this liberty
> > to post my question.
> >
> > My requirement is to develop a kernel/user space module to add an
> > extension to the shell program environment such that this shell forks
> > a mirror look-alike filesystem of the underlying OS to the programs
> > run in that particular shell.
> 
> u seem to be talking about namespaces, if I get you right.
> 
> there is a flag CLONE_NEWNS to the system call 'clone' which does what
> u r talking about.
> 
> RP
> 
> 
> 
> 
> >
> >
> > Was trying to look thru the FAQ and a few list archives to look for
> > ideas around my requirement. The archives were overwhelming.
> >
> >
> > Any ideas/pointers will be a great help,
> > Gracias,
> > decebel
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>

Re: Mirror a file system on the fly

[Ram Pai]

On Thu, 2005-08-18 at 13:27, Dave Schwartz wrote:
> Hi Ram,
> Thanks for the inputs. I was going over the man pages describing the
> clone system call and its option of CLONE_NEWNS. Could understand the
> description only in parts.
> 
> The man page suggests that this flag when set, the cloned child is
> started in a new name space, initialized with a copy of the parent.
> Now does that mean, a program like a shell when cloned with
> CLONE_NEWNS set, will have a copy of file hierarchy of the underlying
> parent process?

Yes the child process will see an exact copy of all the mounts of
various filesystems as that of the parent. However if you mount/unmount
any filesystems in the child, the same will not be mounted/unmounted in
the parent and vice-versa.  Each has its individual view of the
the filesystem heirarchy.

Try the following program that clones off a child process with a mirror
namespace and gives you a bash prompt. Try mounting and unmounting
in this bash prompt and see if the same is visible in a totally
different window.


#include  <stdio.h>
#include  <signal.h>
#include  <sched.h>

char somemem[4096];

int myfunc(){
        system("bash");
}

int
main(int argc, char *argv[])
{
        if(clone(myfunc, somemem, CLONE_NEWNS|SIGCHLD, NULL)) {
                wait(NULL);
        } else {
                printf("clone failed\n");
        }
        printf("exit\n");
}


Hope this helps,
RP




> 
> Gracias,
> decebel
> 
>   
> 
> On 8/19/05, Ram Pai <linuxram@us.ibm.com> wrote:
> > On Thu, 2005-08-18 at 12:40, Dave Schwartz wrote:
> > > Hi list,
> > >
> > > Not too sure if this is the right forum to ask this question but since
> > > my requirement is around linux filesystems, I shall take this liberty
> > > to post my question.
> > >
> > > My requirement is to develop a kernel/user space module to add an
> > > extension to the shell program environment such that this shell forks
> > > a mirror look-alike filesystem of the underlying OS to the programs
> > > run in that particular shell.
> > 
> > u seem to be talking about namespaces, if I get you right.
> > 
> > there is a flag CLONE_NEWNS to the system call 'clone' which does what
> > u r talking about.
> > 
> > RP
> > 
> > 
> > 
> > 
> > >
> > >
> > > Was trying to look thru the FAQ and a few list archives to look for
> > > ideas around my requirement. The archives were overwhelming.
> > >
> > >
> > > Any ideas/pointers will be a great help,
> > > Gracias,
> > > decebel
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> >
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mirror a file system on the fly

[Dave Schwartz]

Ram,

Your code snippet seems to work great as discussed. Thanks. :-)
However, my requirement is slightly different.  What I also want is
that any file created from the mirrored/cloned file-system must not be
available in the parent file system.

Gracias,
decebel


On 8/18/05, Ram Pai <linuxram@us.ibm.com> wrote:
> On Thu, 2005-08-18 at 13:27, Dave Schwartz wrote:
> > Hi Ram,
> > Thanks for the inputs. I was going over the man pages describing the
> > clone system call and its option of CLONE_NEWNS. Could understand the
> > description only in parts.
> >
> > The man page suggests that this flag when set, the cloned child is
> > started in a new name space, initialized with a copy of the parent.
> > Now does that mean, a program like a shell when cloned with
> > CLONE_NEWNS set, will have a copy of file hierarchy of the underlying
> > parent process?
> 
> Yes the child process will see an exact copy of all the mounts of
> various filesystems as that of the parent. However if you mount/unmount
> any filesystems in the child, the same will not be mounted/unmounted in
> the parent and vice-versa.  Each has its individual view of the
> the filesystem heirarchy.
> 
> Try the following program that clones off a child process with a mirror
> namespace and gives you a bash prompt. Try mounting and unmounting
> in this bash prompt and see if the same is visible in a totally
> different window.
> 
> 
> #include  <stdio.h>
> #include  <signal.h>
> #include  <sched.h>
> 
> char somemem[4096];
> 
> int myfunc(){
>         system("bash");
> }
> 
> int
> main(int argc, char *argv[])
> {
>         if(clone(myfunc, somemem, CLONE_NEWNS|SIGCHLD, NULL)) {
>                 wait(NULL);
>         } else {
>                 printf("clone failed\n");
>         }
>         printf("exit\n");
> }
> 
> 
> Hope this helps,
> RP
> 
> 
> 
> 
> >
> > Gracias,
> > decebel
> >
> >
> >
> > On 8/19/05, Ram Pai <linuxram@us.ibm.com> wrote:
> > > On Thu, 2005-08-18 at 12:40, Dave Schwartz wrote:
> > > > Hi list,
> > > >
> > > > Not too sure if this is the right forum to ask this question but since
> > > > my requirement is around linux filesystems, I shall take this liberty
> > > > to post my question.
> > > >
> > > > My requirement is to develop a kernel/user space module to add an
> > > > extension to the shell program environment such that this shell forks
> > > > a mirror look-alike filesystem of the underlying OS to the programs
> > > > run in that particular shell.
> > >
> > > u seem to be talking about namespaces, if I get you right.
> > >
> > > there is a flag CLONE_NEWNS to the system call 'clone' which does what
> > > u r talking about.
> > >
> > > RP
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > Was trying to look thru the FAQ and a few list archives to look for
> > > > ideas around my requirement. The archives were overwhelming.
> > > >
> > > >
> > > > Any ideas/pointers will be a great help,
> > > > Gracias,
> > > > decebel
> > > > -
> > > > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >
> > >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>

Re: Mirror a file system on the fly

[Dave Quigley]

Dave Schwartz wrote:

>Ram,
>
>Your code snippet seems to work great as discussed. Thanks. :-)
>However, my requirement is slightly different.  What I also want is
>that any file created from the mirrored/cloned file-system must not be
>available in the parent file system.
>
>Gracias,
>decebel
>
>
>On 8/18/05, Ram Pai <linuxram@us.ibm.com> wrote:
>  
>
>>On Thu, 2005-08-18 at 13:27, Dave Schwartz wrote:
>>    
>>
>>>Hi Ram,
>>>Thanks for the inputs. I was going over the man pages describing the
>>>clone system call and its option of CLONE_NEWNS. Could understand the
>>>description only in parts.
>>>
>>>The man page suggests that this flag when set, the cloned child is
>>>started in a new name space, initialized with a copy of the parent.
>>>Now does that mean, a program like a shell when cloned with
>>>CLONE_NEWNS set, will have a copy of file hierarchy of the underlying
>>>parent process?
>>>      
>>>
>>Yes the child process will see an exact copy of all the mounts of
>>various filesystems as that of the parent. However if you mount/unmount
>>any filesystems in the child, the same will not be mounted/unmounted in
>>the parent and vice-versa.  Each has its individual view of the
>>the filesystem heirarchy.
>>
>>Try the following program that clones off a child process with a mirror
>>namespace and gives you a bash prompt. Try mounting and unmounting
>>in this bash prompt and see if the same is visible in a totally
>>different window.
>>
>>
>>#include  <stdio.h>
>>#include  <signal.h>
>>#include  <sched.h>
>>
>>char somemem[4096];
>>
>>int myfunc(){
>>        system("bash");
>>}
>>
>>int
>>main(int argc, char *argv[])
>>{
>>        if(clone(myfunc, somemem, CLONE_NEWNS|SIGCHLD, NULL)) {
>>                wait(NULL);
>>        } else {
>>                printf("clone failed\n");
>>        }
>>        printf("exit\n");
>>}
>>
>>
>>Hope this helps,
>>RP
>>
>>
>>
>>
>>    
>>
>>>Gracias,
>>>decebel
>>>
>>>
>>>
>>>On 8/19/05, Ram Pai <linuxram@us.ibm.com> wrote:
>>>      
>>>
>>>>On Thu, 2005-08-18 at 12:40, Dave Schwartz wrote:
>>>>        
>>>>
>>>>>Hi list,
>>>>>
>>>>>Not too sure if this is the right forum to ask this question but since
>>>>>my requirement is around linux filesystems, I shall take this liberty
>>>>>to post my question.
>>>>>
>>>>>My requirement is to develop a kernel/user space module to add an
>>>>>extension to the shell program environment such that this shell forks
>>>>>a mirror look-alike filesystem of the underlying OS to the programs
>>>>>run in that particular shell.
>>>>>          
>>>>>
>>>>u seem to be talking about namespaces, if I get you right.
>>>>
>>>>there is a flag CLONE_NEWNS to the system call 'clone' which does what
>>>>u r talking about.
>>>>
>>>>RP
>>>>
>>>>
>>>>
>>>>
>>>>        
>>>>
>>>>>Was trying to look thru the FAQ and a few list archives to look for
>>>>>ideas around my requirement. The archives were overwhelming.
>>>>>
>>>>>
>>>>>Any ideas/pointers will be a great help,
>>>>>Gracias,
>>>>>decebel
>>>>>-
>>>>>To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>>>>>the body of a message to majordomo@vger.kernel.org
>>>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>          
>>>>>
>>>>        
>>>>
>>>-
>>>To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>>>the body of a message to majordomo@vger.kernel.org
>>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>      
>>>
>>    
>>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>  
>
You might want to look into a project I work on called unionfs It 
contains code for sandboxing which is what your looking to do. The 
current code base is available for both 2.4 and 2.6 so you can do this 
with either kernel version.  You can find the source at 
http://www.filesystems.org/project-unionfs.html the split cache source 
is what you should be looking at. It will give you an example of how to 
create a small sandbox.  If you think unionfs itself can be used for 
your purposes Ide suggest asking on their mailing list..

Dave Quigley