Mapping PIDs from parent->child namespaces

Mapping PIDs from parent->child namespaces

[Mike Heffner]

Hi,

Is it possible for a process running in a parent PID namespace to map 
the PID of a process running in a child's namespace from the 
parent->child's namespace? For example, if I span the process "myproc" 
with CLONE_NEWPID then a call to getpid() inside myproc will return "1" 
whereas in the parent's namespace that process could actually be PID 
"23495". I'd like to be able to know that 23495 maps to 1 in the new NS. 
Obviously, just mapping the first PID is straightforward since I can 
just look at the result of clone(). However, mapping the PIDs of 
processes subsequently forked from "myproc" -- in this example -- I 
haven't been able to figure out.


Thanks,

Mike

Re: Mapping PIDs from parent->child namespaces

[Daniel Lezcano]

On 01/04/2011 12:02 AM, Mike Heffner wrote:
> Hi,
>
> Is it possible for a process running in a parent PID namespace to map
> the PID of a process running in a child's namespace from the
> parent->child's namespace? For example, if I span the process "myproc"
> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
> whereas in the parent's namespace that process could actually be PID
> "23495". I'd like to be able to know that 23495 maps to 1 in the new NS.
> Obviously, just mapping the first PID is straightforward since I can
> just look at the result of clone(). However, mapping the PIDs of
> processes subsequently forked from "myproc" -- in this example -- I
> haven't been able to figure out.

AFAIK, it is not possible.

That would be very nice to show the pid <-> vpid association.

The procfs is a good candidate to show these informations.

That would makes sense to show the content of /proc/<pid>/status with 
the pid relatively to the namespace.

Let me give an example:

Assuming the process '1234' creates a new pid namespace, and the child 
which is '1' in the new namespace has the real pid '4321'. This one 
mounts its /proc.

If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:

...
Tgid:	1
Pid:	1
PPid:	0
...


It could be:

...
Tgid:	4321
Pid:	4321
PPid:	1234
...

as the file is inspected from the parent namespace. Of course, if the 
file is looked from the child namespace context, we will see '1', '1' 
and '0'.

I suppose the patch in the kernel should very small also.

Thoughts ?

Thanks.
   -- Daniel
















































Sauf indication contraire ci-dessus:
Compagnie IBM France
Siège Social : Tour Descartes, 2, avenue Gambetta, La Défense 5, 92400
Courbevoie
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 542.737.118 ?
SIREN/SIRET : 552 118 465 02430

Re: Mapping PIDs from parent->child namespaces

[Cedric Le Goater]

On 01/04/2011 05:04 PM, Daniel Lezcano wrote:
> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>> Hi,
>>
>> Is it possible for a process running in a parent PID namespace to map
>> the PID of a process running in a child's namespace from the
>> parent->child's namespace? For example, if I span the process "myproc"
>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>> whereas in the parent's namespace that process could actually be PID
>> "23495". I'd like to be able to know that 23495 maps to 1 in the new NS.
>> Obviously, just mapping the first PID is straightforward since I can
>> just look at the result of clone(). However, mapping the PIDs of
>> processes subsequently forked from "myproc" -- in this example -- I
>> haven't been able to figure out.
> 
> AFAIK, it is not possible.
> 
> That would be very nice to show the pid <-> vpid association.
> 
> The procfs is a good candidate to show these informations.
> 
> That would makes sense to show the content of /proc/<pid>/status with 
> the pid relatively to the namespace.
> 
> Let me give an example:
> 
> Assuming the process '1234' creates a new pid namespace, and the child 
> which is '1' in the new namespace has the real pid '4321'. This one 
> mounts its /proc.
> 
> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
> 
> ...
> Tgid:	1
> Pid:	1
> PPid:	0
> ...
> 
> 
> It could be:
> 
> ...
> Tgid:	4321
> Pid:	4321
> PPid:	1234
> ...
> 
> as the file is inspected from the parent namespace. Of course, if the 
> file is looked from the child namespace context, we will see '1', '1' 
> and '0'.
> 
> I suppose the patch in the kernel should very small also.
> 
> Thoughts ?

we use the following patch to get the pid of a task as seen from its
pid namespace. It can be useful to identify tasks writing pids in files. 

Cheers,

C.

diff --git a/fs/proc/array.c b/fs/proc/array.c
index fff6572..9a7bfde 100644
--- a/fs/proc/array.c
+++ b/fs/proc/array.c
@@ -337,6 +337,12 @@ static void task_cpus_allowed(struct seq_file *m, struct task
        seq_printf(m, "\n");
 }
 
+static void task_vpid(struct seq_file *m, struct task_struct *task)
+{
+       struct pid_namespace *ns = task_active_pid_ns(task);
+       seq_printf(m, "Vpid:\t%d\n", ns ? task_pid_nr_ns(task, ns) : 0);
+}
+
 int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
                        struct pid *pid, struct task_struct *task)
 {
@@ -357,6 +363,7 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *
        task_show_regs(m, task);
 #endif
        task_context_switch_counts(m, task);
+       task_vpid(m, task);
        return 0;
 }

Re: Mapping PIDs from parent->child namespaces

[Mike Heffner]

On 01/04/2011 11:04 AM, Daniel Lezcano wrote:
> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>> Hi,
>>
>> Is it possible for a process running in a parent PID namespace to map
>> the PID of a process running in a child's namespace from the
>> parent->child's namespace? For example, if I span the process "myproc"
>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>> whereas in the parent's namespace that process could actually be PID
>> "23495". I'd like to be able to know that 23495 maps to 1 in the new NS.
>> Obviously, just mapping the first PID is straightforward since I can
>> just look at the result of clone(). However, mapping the PIDs of
>> processes subsequently forked from "myproc" -- in this example -- I
>> haven't been able to figure out.
>
> AFAIK, it is not possible.
>
> That would be very nice to show the pid<->  vpid association.
>
> The procfs is a good candidate to show these informations.
>
> That would makes sense to show the content of /proc/<pid>/status with
> the pid relatively to the namespace.
>
> Let me give an example:
>
> Assuming the process '1234' creates a new pid namespace, and the child
> which is '1' in the new namespace has the real pid '4321'. This one
> mounts its /proc.
>
> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
>
> ...
> Tgid:	1
> Pid:	1
> PPid:	0
> ...
>
>
> It could be:
>
> ...
> Tgid:	4321
> Pid:	4321
> PPid:	1234
> ...
>
> as the file is inspected from the parent namespace. Of course, if the
> file is looked from the child namespace context, we will see '1', '1'
> and '0'.
>
> I suppose the patch in the kernel should very small also.
>
> Thoughts ?

Would that mean that finding the pid->vpid association for a real PID X 
requires checking all files /proc/<X>/root/proc/<Y>/status where Y is 
all vpids until you find the one where Pid == X? It would be nice to 
have a have a way to check a single file for the association where vpid 
is not known beforehand -- unless I'm misunderstanding your solution.


Cheers,

Mike

Re: Mapping PIDs from parent->child namespaces

[Mike Heffner]

On 01/04/2011 11:44 AM, Cedric Le Goater wrote:
> On 01/04/2011 05:04 PM, Daniel Lezcano wrote:
>> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>>> Hi,
>>>
>>> Is it possible for a process running in a parent PID namespace to map
>>> the PID of a process running in a child's namespace from the
>>> parent->child's namespace? For example, if I span the process "myproc"
>>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>>> whereas in the parent's namespace that process could actually be PID
>>> "23495". I'd like to be able to know that 23495 maps to 1 in the new NS.
>>> Obviously, just mapping the first PID is straightforward since I can
>>> just look at the result of clone(). However, mapping the PIDs of
>>> processes subsequently forked from "myproc" -- in this example -- I
>>> haven't been able to figure out.
>>
>> AFAIK, it is not possible.
>>
>> That would be very nice to show the pid<->  vpid association.
>>
>> The procfs is a good candidate to show these informations.
>>
>> That would makes sense to show the content of /proc/<pid>/status with
>> the pid relatively to the namespace.
>>
>> Let me give an example:
>>
>> Assuming the process '1234' creates a new pid namespace, and the child
>> which is '1' in the new namespace has the real pid '4321'. This one
>> mounts its /proc.
>>
>> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
>>
>> ...
>> Tgid:	1
>> Pid:	1
>> PPid:	0
>> ...
>>
>>
>> It could be:
>>
>> ...
>> Tgid:	4321
>> Pid:	4321
>> PPid:	1234
>> ...
>>
>> as the file is inspected from the parent namespace. Of course, if the
>> file is looked from the child namespace context, we will see '1', '1'
>> and '0'.
>>
>> I suppose the patch in the kernel should very small also.
>>
>> Thoughts ?
>
> we use the following patch to get the pid of a task as seen from its
> pid namespace. It can be useful to identify tasks writing pids in files.
>

Great, I'll try it out. Has there been any interest in getting this into 
the mainline? Are there negatives to advertising child vpid's?


Cheers,

Mike

Re: Mapping PIDs from parent->child namespaces

[Serge Hallyn]

Quoting Mike Heffner (mikeh-ql8fK+M3D0TQT0dZR+AlfA@public.gmane.org):
> On 01/04/2011 11:44 AM, Cedric Le Goater wrote:

> > we use the following patch to get the pid of a task as seen from its
> > pid namespace. It can be useful to identify tasks writing pids in files.
> >
> 
> Great, I'll try it out. Has there been any interest in getting this into 
> the mainline? Are there negatives to advertising child vpid's?

I see no problems offhand with that patch.

Re: Mapping PIDs from parent->child namespaces

[Daniel Lezcano]

On 01/04/2011 05:44 PM, Cedric Le Goater wrote:
> On 01/04/2011 05:04 PM, Daniel Lezcano wrote:
>> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>>> Hi,
>>>
>>> Is it possible for a process running in a parent PID namespace to map
>>> the PID of a process running in a child's namespace from the
>>> parent->child's namespace? For example, if I span the process "myproc"
>>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>>> whereas in the parent's namespace that process could actually be PID
>>> "23495". I'd like to be able to know that 23495 maps to 1 in the new NS.
>>> Obviously, just mapping the first PID is straightforward since I can
>>> just look at the result of clone(). However, mapping the PIDs of
>>> processes subsequently forked from "myproc" -- in this example -- I
>>> haven't been able to figure out.
>> AFAIK, it is not possible.
>>
>> That would be very nice to show the pid<->  vpid association.
>>
>> The procfs is a good candidate to show these informations.
>>
>> That would makes sense to show the content of /proc/<pid>/status with
>> the pid relatively to the namespace.
>>
>> Let me give an example:
>>
>> Assuming the process '1234' creates a new pid namespace, and the child
>> which is '1' in the new namespace has the real pid '4321'. This one
>> mounts its /proc.
>>
>> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
>>
>> ...
>> Tgid:	1
>> Pid:	1
>> PPid:	0
>> ...
>>
>>
>> It could be:
>>
>> ...
>> Tgid:	4321
>> Pid:	4321
>> PPid:	1234
>> ...
>>
>> as the file is inspected from the parent namespace. Of course, if the
>> file is looked from the child namespace context, we will see '1', '1'
>> and '0'.
>>
>> I suppose the patch in the kernel should very small also.
>>
>> Thoughts ?
> we use the following patch to get the pid of a task as seen from its
> pid namespace. It can be useful to identify tasks writing pids in files.
>
> Cheers,
>
> C.
>
> diff --git a/fs/proc/array.c b/fs/proc/array.c
> index fff6572..9a7bfde 100644
> --- a/fs/proc/array.c
> +++ b/fs/proc/array.c
> @@ -337,6 +337,12 @@ static void task_cpus_allowed(struct seq_file *m, struct task
>          seq_printf(m, "\n");
>   }
>
> +static void task_vpid(struct seq_file *m, struct task_struct *task)
> +{
> +       struct pid_namespace *ns = task_active_pid_ns(task);
> +       seq_printf(m, "Vpid:\t%d\n", ns ? task_pid_nr_ns(task, ns) : 0);
> +}
> +
>   int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
>                          struct pid *pid, struct task_struct *task)
>   {
> @@ -357,6 +363,7 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *
>          task_show_regs(m, task);
>   #endif
>          task_context_switch_counts(m, task);
> +       task_vpid(m, task);
>          return 0;
>   }

Right, it is another way to show the vpid but is it enough ? I mean if 
there are multiple pid namespaces, you will see the vpid '2' or whatever 
several times in different /proc/<pid>/status, you will need extra 
information to group these processes to belong to a specific pid 
namespace, no ?

I am also worried about the 'status' file format and if it is allowed to 
add a line in it ... dunno ...

The solution I was proposing is quite similar to this patch, it is just 
the 'task_state' function which is modified to show the pid number 
relatively to the pid namespace which is looking at the file.

If we are the parent pid namespace, we can look at 
/proc/<pid>/root/proc/<vpids>
  * we have the list of the processes with the vpids which are the 
<vpid> filename
  * as we are the parent pid namespace, we look at 
/proc/<pid>/root/proc/<vpid>/status and the pids number are the real pids.
  * if we are in the child pid namespace, we will see the vpid.

It is similar to the /cgroup/tasks file where the pid numbers are 
different depending on what pid namespace we are looking from.

By this way, we don't change the status file format, we have the list of 
the processes belonging to a pid namespace with <vpid> and <pid> and we 
support nested pid namespaces.

Re: Mapping PIDs from parent->child namespaces

[Daniel Lezcano]

On 01/04/2011 08:57 PM, Mike Heffner wrote:
> On 01/04/2011 11:04 AM, Daniel Lezcano wrote:
>> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>>> Hi,
>>>
>>> Is it possible for a process running in a parent PID namespace to map
>>> the PID of a process running in a child's namespace from the
>>> parent->child's namespace? For example, if I span the process "myproc"
>>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>>> whereas in the parent's namespace that process could actually be PID
>>> "23495". I'd like to be able to know that 23495 maps to 1 in the new 
>>> NS.
>>> Obviously, just mapping the first PID is straightforward since I can
>>> just look at the result of clone(). However, mapping the PIDs of
>>> processes subsequently forked from "myproc" -- in this example -- I
>>> haven't been able to figure out.
>>
>> AFAIK, it is not possible.
>>
>> That would be very nice to show the pid<->  vpid association.
>>
>> The procfs is a good candidate to show these informations.
>>
>> That would makes sense to show the content of /proc/<pid>/status with
>> the pid relatively to the namespace.
>>
>> Let me give an example:
>>
>> Assuming the process '1234' creates a new pid namespace, and the child
>> which is '1' in the new namespace has the real pid '4321'. This one
>> mounts its /proc.
>>
>> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
>>
>> ...
>> Tgid:    1
>> Pid:    1
>> PPid:    0
>> ...
>>
>>
>> It could be:
>>
>> ...
>> Tgid:    4321
>> Pid:    4321
>> PPid:    1234
>> ...
>>
>> as the file is inspected from the parent namespace. Of course, if the
>> file is looked from the child namespace context, we will see '1', '1'
>> and '0'.
>>
>> I suppose the patch in the kernel should very small also.
>>
>> Thoughts ?
>
> Would that mean that finding the pid->vpid association for a real PID 
> X requires checking all files /proc/<X>/root/proc/<Y>/status where Y 
> is all vpids until you find the one where Pid == X? It would be nice 
> to have a have a way to check a single file for the association where 
> vpid is not known beforehand -- unless I'm misunderstanding your 
> solution.

Hmm, right. But how do you know a pid is belonging to a specific pid 
namespace ? I mean you can have a single process creating several pid 
namespaces. So while looking at the /proc/<pid>/status, you can see 
several times the same vpid, no ?
I am not sure the kind of informations you want to collect but it is not 
really a problem to build an association table from the userspace by 
browsing the /proc/<pid>/root/proc/<vpids> and their corresponding pid 
from the 'status' file information.

Do you have an example for a pid -> vpid association without looking at 
more informations from /proc ?

Re: Mapping PIDs from parent->child namespaces

[Mike Heffner]

On 01/04/2011 05:13 PM, Daniel Lezcano wrote:
> On 01/04/2011 08:57 PM, Mike Heffner wrote:
>> On 01/04/2011 11:04 AM, Daniel Lezcano wrote:
>>> On 01/04/2011 12:02 AM, Mike Heffner wrote:
>>>> Hi,
>>>>
>>>> Is it possible for a process running in a parent PID namespace to map
>>>> the PID of a process running in a child's namespace from the
>>>> parent->child's namespace? For example, if I span the process "myproc"
>>>> with CLONE_NEWPID then a call to getpid() inside myproc will return "1"
>>>> whereas in the parent's namespace that process could actually be PID
>>>> "23495". I'd like to be able to know that 23495 maps to 1 in the new
>>>> NS.
>>>> Obviously, just mapping the first PID is straightforward since I can
>>>> just look at the result of clone(). However, mapping the PIDs of
>>>> processes subsequently forked from "myproc" -- in this example -- I
>>>> haven't been able to figure out.
>>>
>>> AFAIK, it is not possible.
>>>
>>> That would be very nice to show the pid<-> vpid association.
>>>
>>> The procfs is a good candidate to show these informations.
>>>
>>> That would makes sense to show the content of /proc/<pid>/status with
>>> the pid relatively to the namespace.
>>>
>>> Let me give an example:
>>>
>>> Assuming the process '1234' creates a new pid namespace, and the child
>>> which is '1' in the new namespace has the real pid '4321'. This one
>>> mounts its /proc.
>>>
>>> If the process '1234' looks at /proc/4321/root/proc/1/status, it sees:
>>>
>>> ...
>>> Tgid: 1
>>> Pid: 1
>>> PPid: 0
>>> ...
>>>
>>>
>>> It could be:
>>>
>>> ...
>>> Tgid: 4321
>>> Pid: 4321
>>> PPid: 1234
>>> ...
>>>
>>> as the file is inspected from the parent namespace. Of course, if the
>>> file is looked from the child namespace context, we will see '1', '1'
>>> and '0'.
>>>
>>> I suppose the patch in the kernel should very small also.
>>>
>>> Thoughts ?
>>
>> Would that mean that finding the pid->vpid association for a real PID
>> X requires checking all files /proc/<X>/root/proc/<Y>/status where Y
>> is all vpids until you find the one where Pid == X? It would be nice
>> to have a have a way to check a single file for the association where
>> vpid is not known beforehand -- unless I'm misunderstanding your
>> solution.
>
> Hmm, right. But how do you know a pid is belonging to a specific pid
> namespace ? I mean you can have a single process creating several pid
> namespaces. So while looking at the /proc/<pid>/status, you can see
> several times the same vpid, no ?
> I am not sure the kind of informations you want to collect but it is not
> really a problem to build an association table from the userspace by
> browsing the /proc/<pid>/root/proc/<vpids> and their corresponding pid
> from the 'status' file information.

Yeah, I see how that could be a problem. I guess I was coming from an 
assumption that you knew which namespace a real PID came from just not 
which vpid it was in that namespace.

>
> Do you have an example for a pid -> vpid association without looking at
> more informations from /proc ?
>

I actually did discover another solution. My original requirement was 
for an application monitoring solution where procs in a child namespace 
need to contact an agent running in the root namespace. In this example, 
the agent needs to know the real PID in order to query stats from /proc 
and to know when the process has exited.

I originally tested SO_PEERCRED, but that just returned the vpid. 
However, it does look like this was patched in 2.6.36 with:

http://www.spinics.net/lists/linux-containers/msg20944.html

SO_PEERCRED now returns the realpid in the agent while my application 
can communicate the vpid over the socket.

Re: Mapping PIDs from parent->child namespaces

[Cedric Le Goater]

On 01/04/2011 09:17 PM, Mike Heffner wrote:
>> we use the following patch to get the pid of a task as seen from its
>> pid namespace. It can be useful to identify tasks writing pids in files.
>>
> 
> Great, I'll try it out. Has there been any interest in getting this into 
> the mainline? 

hmm, it has been talked over a few years ago. I can't find the pointer 
anymore. 

> Are there negatives to advertising child vpid's?

I don't think so but the issue is more on gathering clear requirements
for it.

The patch is simple enough to be discussed rapidly and eventually be 
accepted if it is considered useful enough. It does add a user/kernel 
API, which is always a concern. I'm not in favor of exposing too much
of the pid stuff, already overly complex with the nested pid namespace,
but the result of getpid() from a task pov sounds like a legitimate 
information to expose.
   
Why do you need it ? 

 
C.

Re: Mapping PIDs from parent->child namespaces

[Cedric Le Goater]

On 01/05/2011 02:50 PM, Cedric Le Goater wrote:
> Why do you need it ? 

I see you answered that question in another thread.

C.