* Variant symlink filesystem @ 2016-03-11 16:20 Cole 2016-03-11 16:44 ` Austin S. Hemmelgarn 2016-03-11 20:04 ` Richard Weinberger 0 siblings, 2 replies; 19+ messages in thread From: Cole @ 2016-03-11 16:20 UTC (permalink / raw) To: linux-kernel Hi, I have written a Variant Symlink Filesystem for linux, currently implemented as a kernel module: https://github.com/onslauth/varsymfs The code was written for the 3.x kernel. I would like to try to get this included into the linux kernel, and am willing to hand over all copyright and change the license as needed. As such, I would like to know what I can do to try to make this happen. If the code quality or standards are not up to par with those of the linux kernel, or code needs to change due to newer changes introduced into the kernel, please let me know and I will endeavour to make the necessary changes. Please can you also cc me in any replies, as I am not currently subscribed to the list. Thanks /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 16:20 Variant symlink filesystem Cole @ 2016-03-11 16:44 ` Austin S. Hemmelgarn 2016-03-11 20:04 ` Richard Weinberger 1 sibling, 0 replies; 19+ messages in thread From: Austin S. Hemmelgarn @ 2016-03-11 16:44 UTC (permalink / raw) To: Cole, linux-kernel On 2016-03-11 11:20, Cole wrote: > Hi, > > I have written a Variant Symlink Filesystem for linux, currently > implemented as a kernel module: > https://github.com/onslauth/varsymfs > The code was written for the 3.x kernel. > > I would like to try to get this included into the linux kernel, and am > willing to hand over all copyright and change the license as needed. > As such, I would like to know what I can do to try to make this > happen. The first two steps for this to happen are making sure it works on newer kernels (I haven't looked in-depth at your code, but there are not likely to be many changes needed to get it working on 4.4.x assuming you have it working already on late 3.x cycle kernels), and then get it converted to an in-tree module (this also shouldn't be too difficult either, once you figure out which tree to base it on and where to put it in the source tree). Once you get to that point, read through Documentation/SubmittingPatches and follow the steps there. Aside from what's there, you'll probably want to include a couple of well described test cases. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 16:20 Variant symlink filesystem Cole 2016-03-11 16:44 ` Austin S. Hemmelgarn @ 2016-03-11 20:04 ` Richard Weinberger 2016-03-11 20:15 ` Cole 1 sibling, 1 reply; 19+ messages in thread From: Richard Weinberger @ 2016-03-11 20:04 UTC (permalink / raw) To: Cole; +Cc: LKML On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote: > Hi, > > I have written a Variant Symlink Filesystem for linux, currently > implemented as a kernel module: > https://github.com/onslauth/varsymfs > The code was written for the 3.x kernel. > > I would like to try to get this included into the linux kernel, and am > willing to hand over all copyright and change the license as needed. > As such, I would like to know what I can do to try to make this > happen. > > If the code quality or standards are not up to par with those of the > linux kernel, or code needs to change due to newer changes introduced > into the kernel, please let me know and I will endeavour to make the > necessary changes. > > Please can you also cc me in any replies, as I am not currently > subscribed to the list. Why does this need to be a kernel filesystem and not a filesystem in userspace (FUSE)? Especially as you are dealing with environment variables which are owned and controlled by userspace. -- Thanks, //richard ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:04 ` Richard Weinberger @ 2016-03-11 20:15 ` Cole 2016-03-11 20:18 ` Richard Weinberger 0 siblings, 1 reply; 19+ messages in thread From: Cole @ 2016-03-11 20:15 UTC (permalink / raw) To: Richard Weinberger; +Cc: LKML On 11 March 2016 at 22:04, Richard Weinberger <richard.weinberger@gmail.com> wrote: > On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote: >> Hi, >> >> I have written a Variant Symlink Filesystem for linux, currently >> implemented as a kernel module: >> https://github.com/onslauth/varsymfs >> The code was written for the 3.x kernel. >> >> I would like to try to get this included into the linux kernel, and am >> willing to hand over all copyright and change the license as needed. >> As such, I would like to know what I can do to try to make this >> happen. >> >> If the code quality or standards are not up to par with those of the >> linux kernel, or code needs to change due to newer changes introduced >> into the kernel, please let me know and I will endeavour to make the >> necessary changes. >> >> Please can you also cc me in any replies, as I am not currently >> subscribed to the list. > > Why does this need to be a kernel filesystem and not a filesystem in > userspace (FUSE)? > Especially as you are dealing with environment variables which are > owned and controlled > by userspace. The original implementation was in fuse, to prove the concept. However, because we are compiling, as well as running programs and reading/writing files inside of this path, the performance loss is too great. Therefore we moved to this solution. Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:15 ` Cole @ 2016-03-11 20:18 ` Richard Weinberger 2016-03-11 20:22 ` Cole 0 siblings, 1 reply; 19+ messages in thread From: Richard Weinberger @ 2016-03-11 20:18 UTC (permalink / raw) To: Cole; +Cc: LKML Am 11.03.2016 um 21:15 schrieb Cole: > On 11 March 2016 at 22:04, Richard Weinberger > <richard.weinberger@gmail.com> wrote: >> On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote: >>> Hi, >>> >>> I have written a Variant Symlink Filesystem for linux, currently >>> implemented as a kernel module: >>> https://github.com/onslauth/varsymfs >>> The code was written for the 3.x kernel. >>> >>> I would like to try to get this included into the linux kernel, and am >>> willing to hand over all copyright and change the license as needed. >>> As such, I would like to know what I can do to try to make this >>> happen. >>> >>> If the code quality or standards are not up to par with those of the >>> linux kernel, or code needs to change due to newer changes introduced >>> into the kernel, please let me know and I will endeavour to make the >>> necessary changes. >>> >>> Please can you also cc me in any replies, as I am not currently >>> subscribed to the list. >> >> Why does this need to be a kernel filesystem and not a filesystem in >> userspace (FUSE)? >> Especially as you are dealing with environment variables which are >> owned and controlled >> by userspace. > > The original implementation was in fuse, to prove the concept. However, > because we are compiling, as well as running programs and reading/writing > files inside of this path, the performance loss is too great. Therefore we > moved to this solution. Before giving up and going to kernelspace you could try improving the root cause of the performance loss. :) Maybe the kernel interface for finding the env variables can be speeded up. Thanks, //richard ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:18 ` Richard Weinberger @ 2016-03-11 20:22 ` Cole 2016-03-11 20:24 ` Richard Weinberger 0 siblings, 1 reply; 19+ messages in thread From: Cole @ 2016-03-11 20:22 UTC (permalink / raw) To: Richard Weinberger; +Cc: LKML On 11 March 2016 at 22:18, Richard Weinberger <richard@nod.at> wrote: > Am 11.03.2016 um 21:15 schrieb Cole: >> On 11 March 2016 at 22:04, Richard Weinberger >> <richard.weinberger@gmail.com> wrote: >>> On Fri, Mar 11, 2016 at 5:20 PM, Cole <cole@opteqint.net> wrote: >>>> Hi, >>>> >>>> I have written a Variant Symlink Filesystem for linux, currently >>>> implemented as a kernel module: >>>> https://github.com/onslauth/varsymfs >>>> The code was written for the 3.x kernel. >>>> >>>> I would like to try to get this included into the linux kernel, and am >>>> willing to hand over all copyright and change the license as needed. >>>> As such, I would like to know what I can do to try to make this >>>> happen. >>>> >>>> If the code quality or standards are not up to par with those of the >>>> linux kernel, or code needs to change due to newer changes introduced >>>> into the kernel, please let me know and I will endeavour to make the >>>> necessary changes. >>>> >>>> Please can you also cc me in any replies, as I am not currently >>>> subscribed to the list. >>> >>> Why does this need to be a kernel filesystem and not a filesystem in >>> userspace (FUSE)? >>> Especially as you are dealing with environment variables which are >>> owned and controlled >>> by userspace. >> >> The original implementation was in fuse, to prove the concept. However, >> because we are compiling, as well as running programs and reading/writing >> files inside of this path, the performance loss is too great. Therefore we >> moved to this solution. > > Before giving up and going to kernelspace you could try improving the root cause of the performance loss. :) > Maybe the kernel interface for finding the env variables can be speeded up. If I remember correctly, when we were testing the fuse version, we hard coded the path to see if that solved the problem, and the difference between the env lookup code and the hard coded path was almost the same, but substantially slower than the native file system. Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:22 ` Cole @ 2016-03-11 20:24 ` Richard Weinberger 2016-03-11 20:32 ` Cole 2016-03-11 20:38 ` Austin S. Hemmelgarn 0 siblings, 2 replies; 19+ messages in thread From: Richard Weinberger @ 2016-03-11 20:24 UTC (permalink / raw) To: Cole; +Cc: LKML Am 11.03.2016 um 21:22 schrieb Cole: > If I remember correctly, when we were testing the fuse version, we hard coded > the path to see if that solved the problem, and the difference between > the env lookup > code and the hard coded path was almost the same, but substantially slower than > the native file system. And where exactly as the performance problem? Anyway, if you submit your filesystem also provide a decent use case for it. :-) Thanks, //richard ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:24 ` Richard Weinberger @ 2016-03-11 20:32 ` Cole 2016-03-11 20:36 ` Richard Weinberger 2016-03-11 22:27 ` David Lang 2016-03-11 20:38 ` Austin S. Hemmelgarn 1 sibling, 2 replies; 19+ messages in thread From: Cole @ 2016-03-11 20:32 UTC (permalink / raw) To: Richard Weinberger; +Cc: LKML On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote: > Am 11.03.2016 um 21:22 schrieb Cole: >> If I remember correctly, when we were testing the fuse version, we hard coded >> the path to see if that solved the problem, and the difference between >> the env lookup >> code and the hard coded path was almost the same, but substantially slower than >> the native file system. > > And where exactly as the performance problem? > > Anyway, if you submit your filesystem also provide a decent use case for it. :-) Thank you, I will do so. One example as a use case could be to allow for multiple package repositories to exist on a single computer, all in different locations, but with a fixed path so as not to break the package manager, the correct repository then is selected based on ENV variable. That way each user could have their own packages installed that would be separate from the system packages, and no collisions would occur. If you don't mind me asking, are fuse based file systems meant to be as fast or almost as fast as native or in-kernel filesystems? My last experience with them was that they were substantially slower. I also believe with our version of the fuse filesytem that we wrote, the env variable was only being looked up during mount, and then remained static from there onwards. Do you believe that we should have been able to achieve performance almost as good as the in-kernel filesystems? Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:32 ` Cole @ 2016-03-11 20:36 ` Richard Weinberger 2016-03-11 22:27 ` David Lang 1 sibling, 0 replies; 19+ messages in thread From: Richard Weinberger @ 2016-03-11 20:36 UTC (permalink / raw) To: Cole; +Cc: LKML Hi! Am 11.03.2016 um 21:32 schrieb Cole: > On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote: >> Am 11.03.2016 um 21:22 schrieb Cole: >>> If I remember correctly, when we were testing the fuse version, we hard coded >>> the path to see if that solved the problem, and the difference between >>> the env lookup >>> code and the hard coded path was almost the same, but substantially slower than >>> the native file system. >> >> And where exactly as the performance problem? >> >> Anyway, if you submit your filesystem also provide a decent use case for it. :-) > > Thank you, I will do so. One example as a use case could be to allow > for multiple > package repositories to exist on a single computer, all in different > locations, but with > a fixed path so as not to break the package manager, the correct > repository then is > selected based on ENV variable. That way each user could have their own packages > installed that would be separate from the system packages, and no > collisions would > occur. > > If you don't mind me asking, are fuse based file systems meant to be as fast or > almost as fast as native or in-kernel filesystems? My last experience > with them was > that they were substantially slower. I also believe with our version > of the fuse filesytem > that we wrote, the env variable was only being looked up during mount, and then > remained static from there onwards. Do you believe that we should have > been able to > achieve performance almost as good as the in-kernel filesystems? FUSE filesystems are slower. But IMHO your use case cries for FUSE and it does not seem to be very performance critical as all you do is managing a symlink farm and redirecting. IOW all file io can go through the native filesystem. Thanks, //richard ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:32 ` Cole 2016-03-11 20:36 ` Richard Weinberger @ 2016-03-11 22:27 ` David Lang 1 sibling, 0 replies; 19+ messages in thread From: David Lang @ 2016-03-11 22:27 UTC (permalink / raw) To: Cole; +Cc: Richard Weinberger, LKML On Fri, 11 Mar 2016, Cole wrote: > On 11 March 2016 at 22:24, Richard Weinberger <richard@nod.at> wrote: >> Am 11.03.2016 um 21:22 schrieb Cole: >>> If I remember correctly, when we were testing the fuse version, we hard coded >>> the path to see if that solved the problem, and the difference between >>> the env lookup >>> code and the hard coded path was almost the same, but substantially slower than >>> the native file system. >> >> And where exactly as the performance problem? >> >> Anyway, if you submit your filesystem also provide a decent use case for it. :-) > > Thank you, I will do so. One example as a use case could be to allow > for multiple > package repositories to exist on a single computer, all in different > locations, but with > a fixed path so as not to break the package manager, the correct > repository then is > selected based on ENV variable. That way each user could have their own packages > installed that would be separate from the system packages, and no > collisions would > occur. why would this not be a case to use filesystem namespaces and bind mounts? David Lang ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:24 ` Richard Weinberger 2016-03-11 20:32 ` Cole @ 2016-03-11 20:38 ` Austin S. Hemmelgarn 2016-03-11 20:52 ` Cole 1 sibling, 1 reply; 19+ messages in thread From: Austin S. Hemmelgarn @ 2016-03-11 20:38 UTC (permalink / raw) To: Richard Weinberger, Cole; +Cc: LKML On 2016-03-11 15:24, Richard Weinberger wrote: > Am 11.03.2016 um 21:22 schrieb Cole: >> If I remember correctly, when we were testing the fuse version, we hard coded >> the path to see if that solved the problem, and the difference between >> the env lookup >> code and the hard coded path was almost the same, but substantially slower than >> the native file system. > > And where exactly as the performance problem? > > Anyway, if you submit your filesystem also provide a decent use case for it. :-) > I don't know that this qualifies as a use case, but I've seen a number of capability based systems that have a similar concept they usually refer to as 'context dependent symbolic links'. In such cases, the resolution is usually based on what capabilities you posses, and is more of a mapping than a value expansion most of the time, but such usage could be emulated (albeit much less securely) with this. If this could be extended to expand other values (for example, process bit width, or SELinux context, or even what namespace the process is in), it could provide the same functionality almost as securely. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:38 ` Austin S. Hemmelgarn @ 2016-03-11 20:52 ` Cole 2016-03-11 21:51 ` Al Viro 0 siblings, 1 reply; 19+ messages in thread From: Cole @ 2016-03-11 20:52 UTC (permalink / raw) To: Austin S. Hemmelgarn; +Cc: Richard Weinberger, LKML On 11 March 2016 at 22:38, Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote: > On 2016-03-11 15:24, Richard Weinberger wrote: >> >> Am 11.03.2016 um 21:22 schrieb Cole: >>> >>> If I remember correctly, when we were testing the fuse version, we hard >>> coded >>> the path to see if that solved the problem, and the difference between >>> the env lookup >>> code and the hard coded path was almost the same, but substantially >>> slower than >>> the native file system. >> >> >> And where exactly as the performance problem? >> >> Anyway, if you submit your filesystem also provide a decent use case for >> it. :-) >> > I don't know that this qualifies as a use case, but I've seen a number of > capability based systems that have a similar concept they usually refer to > as 'context dependent symbolic links'. In such cases, the resolution is > usually based on what capabilities you posses, and is more of a mapping than > a value expansion most of the time, but such usage could be emulated (albeit > much less securely) with this. If this could be extended to expand other > values (for example, process bit width, or SELinux context, or even what > namespace the process is in), it could provide the same functionality almost > as securely. The implementation doesn't necessarily have to continue to work with env variables. On FreeBSD, the variant symlinks function by using variables stored in kernel memory, and have a hierarchical lookup, starting with user defined values and terminating with global entries. I am not aware of such functionality existing on linux, but if someone could point me at something similar to that, I would much prefer to use that, as there are issues with variables that are exported or modified during process execution. Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 20:52 ` Cole @ 2016-03-11 21:51 ` Al Viro 2016-03-11 22:03 ` Cole 0 siblings, 1 reply; 19+ messages in thread From: Al Viro @ 2016-03-11 21:51 UTC (permalink / raw) To: Cole; +Cc: Austin S. Hemmelgarn, Richard Weinberger, LKML On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote: > The implementation doesn't necessarily have to continue to work with > env variables. On FreeBSD, the variant symlinks function by using > variables stored in kernel memory, and have a hierarchical lookup, > starting with user defined values and terminating with global entries. > I am not aware of such functionality existing on linux, but if someone > could point me at something similar to that, I would much prefer to > use that, as there are issues with variables that are exported or > modified during process execution. Put your processes into a separate namespace and use mount --bind in it... ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 21:51 ` Al Viro @ 2016-03-11 22:03 ` Cole 2016-03-11 22:24 ` Al Viro 2016-03-11 22:38 ` David Lang 0 siblings, 2 replies; 19+ messages in thread From: Cole @ 2016-03-11 22:03 UTC (permalink / raw) To: Al Viro; +Cc: LKML, Austin S. Hemmelgarn, Richard Weinberger On 11 March 2016 at 23:51, Al Viro <viro@zeniv.linux.org.uk> wrote: > On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote: > >> The implementation doesn't necessarily have to continue to work with >> env variables. On FreeBSD, the variant symlinks function by using >> variables stored in kernel memory, and have a hierarchical lookup, >> starting with user defined values and terminating with global entries. >> I am not aware of such functionality existing on linux, but if someone >> could point me at something similar to that, I would much prefer to >> use that, as there are issues with variables that are exported or >> modified during process execution. > > Put your processes into a separate namespace and use mount --bind in it... This was one of the first solutions we looked at, and using various namespaces. However we would like to be able to have multiple terminal sessions open, and be able to have each session using a different mount point, or be able to use the other terminals mount point, i.e. switching the mount point to that of the other terminals. We would also like the shell to be able to make use of these, and use shell commands such as 'ls'. When we originally looked at namespaces and containers, we could not find a solution to achieve the above. Is this possible using namespaces? Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 22:03 ` Cole @ 2016-03-11 22:24 ` Al Viro 2016-03-11 22:31 ` Cole 2016-03-11 22:38 ` David Lang 1 sibling, 1 reply; 19+ messages in thread From: Al Viro @ 2016-03-11 22:24 UTC (permalink / raw) To: Cole; +Cc: LKML, Austin S. Hemmelgarn, Richard Weinberger On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote: > This was one of the first solutions we looked at, and using various > namespaces. However we would like to be able to have multiple terminal > sessions open, and be able to have each session using a different > mount point, or be able to use the other terminals mount point, i.e. > switching the mount point to that of the other terminals. We would > also like the shell to be able to make use of these, and use shell > commands such as 'ls'. > > When we originally looked at namespaces and containers, we could not > find a solution to achieve the above. Is this possible using > namespaces? I'd try to look at setns(2) if you want processes joinging existing namespaces. I'm afraid that I'll need to get some sleep before I'll be up to asking the right questions for figuring out what requirements do you have and what's the best way to do it - after a while coffee stops being efficient and I'm already several hours past that ;-/ ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 22:24 ` Al Viro @ 2016-03-11 22:31 ` Cole 2016-03-11 22:48 ` David Lang 0 siblings, 1 reply; 19+ messages in thread From: Cole @ 2016-03-11 22:31 UTC (permalink / raw) To: Al Viro; +Cc: LKML, Austin S. Hemmelgarn, Richard Weinberger On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote: > On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote: > >> This was one of the first solutions we looked at, and using various >> namespaces. However we would like to be able to have multiple terminal >> sessions open, and be able to have each session using a different >> mount point, or be able to use the other terminals mount point, i.e. >> switching the mount point to that of the other terminals. We would >> also like the shell to be able to make use of these, and use shell >> commands such as 'ls'. >> >> When we originally looked at namespaces and containers, we could not >> find a solution to achieve the above. Is this possible using >> namespaces? > > I'd try to look at setns(2) if you want processes joinging existing namespaces. > I'm afraid that I'll need to get some sleep before I'll be up to asking > the right questions for figuring out what requirements do you have and > what's the best way to do it - after a while coffee stops being efficient > and I'm already several hours past that ;-/ Sure, not a problem, when you have time to reply I will gladly welcome any feed back. As for the usage, I'll explain it a bit so that you have something to work off of when you get a chance to read it. The problem we encountered with namespaces when we looked at it more than a year ago was 'how do you get the shell' to join them, or into one. And also how do you move the shell in one terminal session into a namespace that another shell is currently in. We wanted a solution that doesn't require modifying existing programs to make them namespace aware. However, as I said, this was more than a year ago that we looked at it, and we could easily have misunderstood something, or not understood the full functionality available. If you say this is possible, without modifying programs such as bash, could you please point me in the direction of the documentation describing this, and I will try to educate myself. Thanks /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 22:31 ` Cole @ 2016-03-11 22:48 ` David Lang 2016-03-11 22:54 ` Cole 0 siblings, 1 reply; 19+ messages in thread From: David Lang @ 2016-03-11 22:48 UTC (permalink / raw) To: Cole; +Cc: Al Viro, LKML, Austin S. Hemmelgarn, Richard Weinberger On Sat, 12 Mar 2016, Cole wrote: > On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote: >> On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote: >> >>> This was one of the first solutions we looked at, and using various >>> namespaces. However we would like to be able to have multiple terminal >>> sessions open, and be able to have each session using a different >>> mount point, or be able to use the other terminals mount point, i.e. >>> switching the mount point to that of the other terminals. We would >>> also like the shell to be able to make use of these, and use shell >>> commands such as 'ls'. >>> >>> When we originally looked at namespaces and containers, we could not >>> find a solution to achieve the above. Is this possible using >>> namespaces? >> >> I'd try to look at setns(2) if you want processes joinging existing namespaces. >> I'm afraid that I'll need to get some sleep before I'll be up to asking >> the right questions for figuring out what requirements do you have and >> what's the best way to do it - after a while coffee stops being efficient >> and I'm already several hours past that ;-/ > > > Sure, not a problem, when you have time to reply I will gladly welcome > any feed back. > > As for the usage, I'll explain it a bit so that you have something to > work off of when you get a chance to read it. > > The problem we encountered with namespaces when we looked at it more > than a year ago was 'how do you get the shell' to join them, or into > one. And also how do you move the shell in one terminal session into a > namespace that another shell is currently in. We wanted a solution > that doesn't require modifying existing programs to make them > namespace aware. However, as I said, this was more than a year ago > that we looked at it, and we could easily have misunderstood > something, or not understood the full functionality available. If you > say this is possible, without modifying programs such as bash, could > you please point me in the direction of the documentation describing > this, and I will try to educate myself. looking at the setns() function, it seems like you could have a suid helper program that you run in one session that changes the namespace and then invokes a bash shell in that namespace that you then run unmodified stuff in. it seems like there should be a way for a root program to change the namespace of another, but I'm not finding it at the moment. There is the nsenter program that will run a program inside an existing namespace. It looks like you need something similar that implements some permission checking (only let you go into namespaces of other programs for the same user or similar), but you should be able to make proof-of-concept scripts with nsenter. David Lang ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 22:48 ` David Lang @ 2016-03-11 22:54 ` Cole 0 siblings, 0 replies; 19+ messages in thread From: Cole @ 2016-03-11 22:54 UTC (permalink / raw) To: David Lang; +Cc: Al Viro, LKML, Austin S. Hemmelgarn, Richard Weinberger On 12 March 2016 at 00:48, David Lang <david@lang.hm> wrote: > On Sat, 12 Mar 2016, Cole wrote: > >> On 12 March 2016 at 00:24, Al Viro <viro@zeniv.linux.org.uk> wrote: >>> >>> On Sat, Mar 12, 2016 at 12:03:11AM +0200, Cole wrote: >>> >>>> This was one of the first solutions we looked at, and using various >>>> namespaces. However we would like to be able to have multiple terminal >>>> sessions open, and be able to have each session using a different >>>> mount point, or be able to use the other terminals mount point, i.e. >>>> switching the mount point to that of the other terminals. We would >>>> also like the shell to be able to make use of these, and use shell >>>> commands such as 'ls'. >>>> >>>> When we originally looked at namespaces and containers, we could not >>>> find a solution to achieve the above. Is this possible using >>>> namespaces? >>> >>> >>> I'd try to look at setns(2) if you want processes joinging existing >>> namespaces. >>> I'm afraid that I'll need to get some sleep before I'll be up to asking >>> the right questions for figuring out what requirements do you have and >>> what's the best way to do it - after a while coffee stops being efficient >>> and I'm already several hours past that ;-/ >> >> >> >> Sure, not a problem, when you have time to reply I will gladly welcome >> any feed back. >> >> As for the usage, I'll explain it a bit so that you have something to >> work off of when you get a chance to read it. >> >> The problem we encountered with namespaces when we looked at it more >> than a year ago was 'how do you get the shell' to join them, or into >> one. And also how do you move the shell in one terminal session into a >> namespace that another shell is currently in. We wanted a solution >> that doesn't require modifying existing programs to make them >> namespace aware. However, as I said, this was more than a year ago >> that we looked at it, and we could easily have misunderstood >> something, or not understood the full functionality available. If you >> say this is possible, without modifying programs such as bash, could >> you please point me in the direction of the documentation describing >> this, and I will try to educate myself. > > > looking at the setns() function, it seems like you could have a suid helper > program that you run in one session that changes the namespace and then > invokes a bash shell in that namespace that you then run unmodified stuff > in. > > it seems like there should be a way for a root program to change the > namespace of another, but I'm not finding it at the moment. > > There is the nsenter program that will run a program inside an existing > namespace. It looks like you need something similar that implements some > permission checking (only let you go into namespaces of other programs for > the same user or similar), but you should be able to make proof-of-concept > scripts with nsenter. Thank you very much for the explanation and links. If I understand you correctly, you have described something similar to how docker functions, with a single daemon that runs, and the various support binaries make calls to it to perform the necessary functions. The only concern as you mentioned is that there does not seem to be any functionality available to move a process from one namespace to another. However, the only process in my design that would need to do that would be the shell, so nsenter might be sufficient. I will look into your suggestions, as it then means I do not need to have anything inlined into the kernel. Thank you for all the help and advice. Regards /Cole ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Variant symlink filesystem 2016-03-11 22:03 ` Cole 2016-03-11 22:24 ` Al Viro @ 2016-03-11 22:38 ` David Lang 1 sibling, 0 replies; 19+ messages in thread From: David Lang @ 2016-03-11 22:38 UTC (permalink / raw) To: Cole; +Cc: Al Viro, LKML, Austin S. Hemmelgarn, Richard Weinberger On Sat, 12 Mar 2016, Cole wrote: > On 11 March 2016 at 23:51, Al Viro <viro@zeniv.linux.org.uk> wrote: >> On Fri, Mar 11, 2016 at 10:52:52PM +0200, Cole wrote: >> >>> The implementation doesn't necessarily have to continue to work with >>> env variables. On FreeBSD, the variant symlinks function by using >>> variables stored in kernel memory, and have a hierarchical lookup, >>> starting with user defined values and terminating with global entries. >>> I am not aware of such functionality existing on linux, but if someone >>> could point me at something similar to that, I would much prefer to >>> use that, as there are issues with variables that are exported or >>> modified during process execution. >> >> Put your processes into a separate namespace and use mount --bind in it... > > This was one of the first solutions we looked at, and using various > namespaces. However we would like to be able to have multiple terminal > sessions open, and be able to have each session using a different > mount point, or be able to use the other terminals mount point, i.e. > switching the mount point to that of the other terminals. We would > also like the shell to be able to make use of these, and use shell > commands such as 'ls'. you should be able to have multiple sessions using the same namespace. There is the lwn.net series on namespaces at https://lwn.net/Articles/531114/ from what I'm looking at, this should be possible with the right mount options. It's not as trivial as setting an environment variable, but if it's all scripted, that shouldn't matter to the user. you would need to use the setns() call to have one session join an existing namespace rather than creating a new one. now, changing namespaces does require CAP_SYS_ADMIN, so if you are not running things as root, you may need to create a small daemon to run as root that reassigns your different sessions from one ns to another. David Lang > When we originally looked at namespaces and containers, we could not > find a solution to achieve the above. Is this possible using > namespaces? > > Regards > /Cole > ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2016-03-11 22:54 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-03-11 16:20 Variant symlink filesystem Cole 2016-03-11 16:44 ` Austin S. Hemmelgarn 2016-03-11 20:04 ` Richard Weinberger 2016-03-11 20:15 ` Cole 2016-03-11 20:18 ` Richard Weinberger 2016-03-11 20:22 ` Cole 2016-03-11 20:24 ` Richard Weinberger 2016-03-11 20:32 ` Cole 2016-03-11 20:36 ` Richard Weinberger 2016-03-11 22:27 ` David Lang 2016-03-11 20:38 ` Austin S. Hemmelgarn 2016-03-11 20:52 ` Cole 2016-03-11 21:51 ` Al Viro 2016-03-11 22:03 ` Cole 2016-03-11 22:24 ` Al Viro 2016-03-11 22:31 ` Cole 2016-03-11 22:48 ` David Lang 2016-03-11 22:54 ` Cole 2016-03-11 22:38 ` David Lang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.