Use cases for multiple uid mapping?

Use cases for multiple uid mapping?

[Eric W. Biederman]

We had a discussion in the hackroom at LPC talking about use cases for
a shiftfs style setup where there are different mappings of uids to
disk.

In the discussion we had a couple of ideas of kernel developments
we should look at that address some of these.

- Fix rlimits in user namespaces (This potentially allows multiple
  containers to run with the same userids simplifying the mapping
  problem).

- Look at extending kuid_t to 64bits and using the highbits to
  implement uids that are private to user namespaces and don't
  map out.
  
- Look at ways for allowing setgroups unprivileged.


Together this has the potential that the existing uid & gid mappings
will be able to function the same as the proposed fusid mappings. Fingers crossed.


I had some problems with audio and a lot of people were talking
quickly.  So I did not manage to capture everyone's use cases.   And I
definitely was not able to see how everyone's use cases interacted with
the changes we are looking at.

I know for certain I missed Serge's usecase (apologies).

Can people follow up to this and report their use cases?

There are some real challenges and I would like to see if we
can solve them, while avoiding scary problems like changing
uids on write.

Eric







  


_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Stéphane Graber via Containers]

On Fri, Aug 28, 2020 at 11:21 AM Eric W. Biederman
<ebiederm@xmission.com> wrote:
>
>
> We had a discussion in the hackroom at LPC talking about use cases for
> a shiftfs style setup where there are different mappings of uids to
> disk.
>
> In the discussion we had a couple of ideas of kernel developments
> we should look at that address some of these.
>
> - Fix rlimits in user namespaces (This potentially allows multiple
>   containers to run with the same userids simplifying the mapping
>   problem).
>
> - Look at extending kuid_t to 64bits and using the highbits to
>   implement uids that are private to user namespaces and don't
>   map out.
>
> - Look at ways for allowing setgroups unprivileged.
>
>
> Together this has the potential that the existing uid & gid mappings
> will be able to function the same as the proposed fusid mappings. Fingers crossed.
>
>
> I had some problems with audio and a lot of people were talking
> quickly.  So I did not manage to capture everyone's use cases.   And I
> definitely was not able to see how everyone's use cases interacted with
> the changes we are looking at.
>
> I know for certain I missed Serge's usecase (apologies).
>
> Can people follow up to this and report their use cases?
>
> There are some real challenges and I would like to see if we
> can solve them, while avoiding scary problems like changing
> uids on write.
>
> Eric

Hi Eric,

It was great to have you and everyone else participate in that session.
There was indeed a lot covered in there and a lot of use cases we'll
have to keep in mind!

The main outcomes we captured I believe were:
 - Fixing rlimits in user namespaces such that one namespace cannot
affect another
 - Consider switching the kernel uid/git types to 64bit, allowing for
a hidden 32bit identifier and fully usable 32bit uid/gid range for the
container
 - Find a way to allow setgroups() in a user namespace while keeping
in mind the case of groups used for negative access control
 - Add optional enforcement that overflow uid/gid as set through
sysctl cannot be used as regular uid/gid on the system

Christian is working on a comprehensive summary of the session based
on the shared notes document we circulated at the beginning of the
session which will try to separate the different topics and cover use
cases and potential solutions for each as we discussed in the BoF.

We'll make sure that everyone who attended the BoF gets CCed on that,
on top of the usual mailing-lists so that should any of the use raised
use cases have been missed in the notes, those folks can raise them
again so we don't end up missing anything :)

Stéphane
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Sargun Dhillon]

On Fri, Aug 28, 2020 at 8:56 AM Stéphane Graber via Containers
<containers@lists.linux-foundation.org> wrote:
>
> On Fri, Aug 28, 2020 at 11:21 AM Eric W. Biederman
> <ebiederm@xmission.com> wrote:
> >
> >
> > We had a discussion in the hackroom at LPC talking about use cases for
> > a shiftfs style setup where there are different mappings of uids to
> > disk.
> >
> > In the discussion we had a couple of ideas of kernel developments
> > we should look at that address some of these.
> >
> > - Fix rlimits in user namespaces (This potentially allows multiple
> >   containers to run with the same userids simplifying the mapping
> >   problem).
> >
> > - Look at extending kuid_t to 64bits and using the highbits to
> >   implement uids that are private to user namespaces and don't
> >   map out.
> >
> > - Look at ways for allowing setgroups unprivileged.
> >
> >
> > Together this has the potential that the existing uid & gid mappings
> > will be able to function the same as the proposed fusid mappings. Fingers crossed.
> >
> >
> > I had some problems with audio and a lot of people were talking
> > quickly.  So I did not manage to capture everyone's use cases.   And I
> > definitely was not able to see how everyone's use cases interacted with
> > the changes we are looking at.
> >
> > I know for certain I missed Serge's usecase (apologies).
> >
> > Can people follow up to this and report their use cases?
> >
> > There are some real challenges and I would like to see if we
> > can solve them, while avoiding scary problems like changing
> > uids on write.
> >
> > Eric
>
> Hi Eric,
>
> It was great to have you and everyone else participate in that session.
> There was indeed a lot covered in there and a lot of use cases we'll
> have to keep in mind!
>
> The main outcomes we captured I believe were:
>  - Fixing rlimits in user namespaces such that one namespace cannot
> affect another
>  - Consider switching the kernel uid/git types to 64bit, allowing for
> a hidden 32bit identifier and fully usable 32bit uid/gid range for the
> container
Seconding this. We run into a lot of problems with high UIDs. Some
systems think that uid 2**31-1 (I think it's 2**31-1) is their version of
'nobody'. Being able to have 64-bit UIDs would solve this.

Also making sure this works with NFSv4 (both ID mapped, and non-
ID mapped) is very important to us. It's fine if the UID / GID is just
"passthrough", and we don't need different mappings between
different filesystems in NFS (for now). We would like to be able
to be able to run idmapper in the user / pid / net namespace of
the container.

>  - Find a way to allow setgroups() in a user namespace while keeping
> in mind the case of groups used for negative access control
>  - Add optional enforcement that overflow uid/gid as set through
> sysctl cannot be used as regular uid/gid on the system
>
> Christian is working on a comprehensive summary of the session based
> on the shared notes document we circulated at the beginning of the
> session which will try to separate the different topics and cover use
> cases and potential solutions for each as we discussed in the BoF.
>
> We'll make sure that everyone who attended the BoF gets CCed on that,
> on top of the usual mailing-lists so that should any of the use raised
> use cases have been missed in the notes, those folks can raise them
> again so we don't end up missing anything :)
>
> Stéphane
> _______________________________________________
> Containers mailing list
> Containers@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[James Bottomley]

On Fri, 2020-08-28 at 10:17 -0500, Eric W. Biederman wrote:
[...]
> Can people follow up to this and report their use cases?
> 
> There are some real challenges and I would like to see if we
> can solve them, while avoiding scary problems like changing
> uids on write.

The main problem shiftfs has that fsuid doesn't is write at privileged
id, so mainly safely writing to kuid 0.

shiftfs tried to solve this by exposing the shifted mount point safely,
but maybe there's another way to do it.

James

_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Serge E. Hallyn]

On Fri, Aug 28, 2020 at 10:17:16AM -0500, Eric W. Biederman wrote:
> 
> We had a discussion in the hackroom at LPC talking about use cases for
> a shiftfs style setup where there are different mappings of uids to
> disk.
> 
> In the discussion we had a couple of ideas of kernel developments
> we should look at that address some of these.
> 
> - Fix rlimits in user namespaces (This potentially allows multiple
>   containers to run with the same userids simplifying the mapping
>   problem).
> 
> - Look at extending kuid_t to 64bits and using the highbits to
>   implement uids that are private to user namespaces and don't
>   map out.
>   
> - Look at ways for allowing setgroups unprivileged.
> 
> 
> Together this has the potential that the existing uid & gid mappings
> will be able to function the same as the proposed fusid mappings. Fingers crossed.
> 
> 
> I had some problems with audio and a lot of people were talking
> quickly.  So I did not manage to capture everyone's use cases.   And I
> definitely was not able to see how everyone's use cases interacted with
> the changes we are looking at.
> 
> I know for certain I missed Serge's usecase (apologies).
> 
> Can people follow up to this and report their use cases?

Sorry - I'll do so later this week.

> There are some real challenges and I would like to see if we
> can solve them, while avoiding scary problems like changing
> uids on write.
> 
> Eric
> 
> 
> 
> 
> 
> 
> 
>   
> 
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Eric W. Biederman]

"Serge E. Hallyn" <serge@hallyn.com> writes:

> On Fri, Aug 28, 2020 at 10:17:16AM -0500, Eric W. Biederman wrote:
>> 
>> We had a discussion in the hackroom at LPC talking about use cases for
>> a shiftfs style setup where there are different mappings of uids to
>> disk.
>> 
>> In the discussion we had a couple of ideas of kernel developments
>> we should look at that address some of these.
>> 
>> - Fix rlimits in user namespaces (This potentially allows multiple
>>   containers to run with the same userids simplifying the mapping
>>   problem).
>> 
>> - Look at extending kuid_t to 64bits and using the highbits to
>>   implement uids that are private to user namespaces and don't
>>   map out.
>>   
>> - Look at ways for allowing setgroups unprivileged.
>> 
>> 
>> Together this has the potential that the existing uid & gid mappings
>> will be able to function the same as the proposed fusid mappings. Fingers crossed.
>> 
>> 
>> I had some problems with audio and a lot of people were talking
>> quickly.  So I did not manage to capture everyone's use cases.   And I
>> definitely was not able to see how everyone's use cases interacted with
>> the changes we are looking at.
>> 
>> I know for certain I missed Serge's usecase (apologies).
>> 
>> Can people follow up to this and report their use cases?
>
> Sorry - I'll do so later this week.

Thank you.

I know we have the OCI use case of overlayfs and sharing storage
between containers.

I know we have the lxc case of not wanting to be strangled by ulimits.
So not using the same uid between containers even when it is logically
the same users.

I know the brainstorming was going a lot of different directions and I
piped up and said that we should probably focus on handling the stranger
cases with fuse mounts, and the other capabilities we have now.

It really will be valuable to understand the other cases so we don't
code ourselves into a corner that only works for the most vocal of the
developers.

Eric
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Alban Crequy]

On Tue, 1 Sep 2020 at 16:53, Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> "Serge E. Hallyn" <serge@hallyn.com> writes:
>
> > On Fri, Aug 28, 2020 at 10:17:16AM -0500, Eric W. Biederman wrote:
> >>
> >> We had a discussion in the hackroom at LPC talking about use cases for
> >> a shiftfs style setup where there are different mappings of uids to
> >> disk.
> >>
> >> In the discussion we had a couple of ideas of kernel developments
> >> we should look at that address some of these.
> >>
> >> - Fix rlimits in user namespaces (This potentially allows multiple
> >>   containers to run with the same userids simplifying the mapping
> >>   problem).
> >>
> >> - Look at extending kuid_t to 64bits and using the highbits to
> >>   implement uids that are private to user namespaces and don't
> >>   map out.
> >>
> >> - Look at ways for allowing setgroups unprivileged.
> >>
> >>
> >> Together this has the potential that the existing uid & gid mappings
> >> will be able to function the same as the proposed fusid mappings. Fingers crossed.
> >>
> >>
> >> I had some problems with audio and a lot of people were talking
> >> quickly.  So I did not manage to capture everyone's use cases.   And I
> >> definitely was not able to see how everyone's use cases interacted with
> >> the changes we are looking at.
> >>
> >> I know for certain I missed Serge's usecase (apologies).
> >>
> >> Can people follow up to this and report their use cases?
> >
> > Sorry - I'll do so later this week.
>
> Thank you.
>
> I know we have the OCI use case of overlayfs and sharing storage
> between containers.

Is there a description of this use case, and ideas of possible solutions?

There was a use case that was brought to a recent Kubernetes sig-node
meeting, but I am not sure it is the same one, so I'll describe this
one.

We're working to enable user namespaces in Kubernetes with two possible setups:
1. Each Kubernetes pod has its own userns but with the same user id
mapping (to make it easy to manage shared volumes)
2. Each Kubernetes pod has its own userns with non-overlapping user id
mapping (providing additional isolation between pods)

Pods could be executed with a Kubernetes sidecar such as Envoy,
meaning that if there are 'n' pods running on a node, there will be
'n' containers running the Envoy container image. In the second setup,
each Envoy container will have a different user id mapping.

The Envoy container image will be snapshotted in as many copies, and
each copy will be chown'ed appropriately for each non-overlapping user
id mapping. Each copy will then be used as the lowerdir for the
overlayfs rootfs of the container.

This results in a waste of disk space. Without user namespaces, we
don't have this problem, since the same directory can be used as
lowerdir for each container's overlayfs rootfs.

This is not specific to sidecars, but they exacerbate the problem.

I was not present in the Plumbers discussions, so I don't know if
possible solutions were discussed. For example: could overlayfs get a
mount option to specify an id mapping to apply on the lowerdir? In the
case of containerd + runc, the mount option would probably need to be
an id mapping rather than a reference to an existing user namespace,
because the overlayfs mount is set up by containerd before runc is
executed, so before the user namespace is unshared.

Alban

> I know we have the lxc case of not wanting to be strangled by ulimits.
> So not using the same uid between containers even when it is logically
> the same users.
>
> I know the brainstorming was going a lot of different directions and I
> piped up and said that we should probably focus on handling the stranger
> cases with fuse mounts, and the other capabilities we have now.
>
> It really will be valuable to understand the other cases so we don't
> code ourselves into a corner that only works for the most vocal of the
> developers.
>
> Eric
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

Re: Use cases for multiple uid mapping?

[Christian Brauner]

On Thu, Sep 24, 2020 at 07:09:21PM +0200, Alban Crequy wrote:
> On Tue, 1 Sep 2020 at 16:53, Eric W. Biederman <ebiederm@xmission.com> wrote:
> >
> > "Serge E. Hallyn" <serge@hallyn.com> writes:
> >
> > > On Fri, Aug 28, 2020 at 10:17:16AM -0500, Eric W. Biederman wrote:
> > >>
> > >> We had a discussion in the hackroom at LPC talking about use cases for
> > >> a shiftfs style setup where there are different mappings of uids to
> > >> disk.
> > >>
> > >> In the discussion we had a couple of ideas of kernel developments
> > >> we should look at that address some of these.
> > >>
> > >> - Fix rlimits in user namespaces (This potentially allows multiple
> > >>   containers to run with the same userids simplifying the mapping
> > >>   problem).
> > >>
> > >> - Look at extending kuid_t to 64bits and using the highbits to
> > >>   implement uids that are private to user namespaces and don't
> > >>   map out.
> > >>
> > >> - Look at ways for allowing setgroups unprivileged.
> > >>
> > >>
> > >> Together this has the potential that the existing uid & gid mappings
> > >> will be able to function the same as the proposed fusid mappings. Fingers crossed.
> > >>
> > >>
> > >> I had some problems with audio and a lot of people were talking
> > >> quickly.  So I did not manage to capture everyone's use cases.   And I
> > >> definitely was not able to see how everyone's use cases interacted with
> > >> the changes we are looking at.
> > >>
> > >> I know for certain I missed Serge's usecase (apologies).
> > >>
> > >> Can people follow up to this and report their use cases?
> > >
> > > Sorry - I'll do so later this week.
> >
> > Thank you.
> >
> > I know we have the OCI use case of overlayfs and sharing storage
> > between containers.
> 
> Is there a description of this use case, and ideas of possible solutions?
> 
> There was a use case that was brought to a recent Kubernetes sig-node
> meeting, but I am not sure it is the same one, so I'll describe this
> one.
> 
> We're working to enable user namespaces in Kubernetes with two possible setups:
> 1. Each Kubernetes pod has its own userns but with the same user id
> mapping (to make it easy to manage shared volumes)
> 2. Each Kubernetes pod has its own userns with non-overlapping user id
> mapping (providing additional isolation between pods)
> 
> Pods could be executed with a Kubernetes sidecar such as Envoy,
> meaning that if there are 'n' pods running on a node, there will be
> 'n' containers running the Envoy container image. In the second setup,
> each Envoy container will have a different user id mapping.
> 
> The Envoy container image will be snapshotted in as many copies, and
> each copy will be chown'ed appropriately for each non-overlapping user
> id mapping. Each copy will then be used as the lowerdir for the
> overlayfs rootfs of the container.
> 
> This results in a waste of disk space. Without user namespaces, we
> don't have this problem, since the same directory can be used as
> lowerdir for each container's overlayfs rootfs.
> 
> This is not specific to sidecars, but they exacerbate the problem.
> 
> I was not present in the Plumbers discussions, so I don't know if
> possible solutions were discussed. For example: could overlayfs get a

Hey Alban,

Tycho, Aleksa, Serge and I are collaborating on a patchset that intends
to cover all of these use-cases and that we hope to have ready for
review early after the next merge window.

Christian
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers