Re: [PATCH] processes with shared vm

Re: [PATCH] processes with shared vm

[Dave McCracken]

--On Friday, August 17, 2001 14:30:28 +0000 Mark Hahn 
<hahn@physics.mcmaster.ca> wrote:

> from a quick glance, there is no tgid, and deliberately so:
> proc.thread_group is a list of siblings, no?  siblings that
> normally share their struct mm.

There is in fact a tgid in the task_struct.  Its value is the pid of the 
task that first called clone() with CLONE_THREAD set.  And in fact the 
getpid() system call actually returns tgid instead of pid.  It seems to me 
only reasonable to include this in the information available via /proc.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [PATCH] processes with shared vm

[Andi Kleen]

Terje Eggestad <terje.eggestad@scali.no> writes:

> I figured out that it's difficult to find out from /proc
> which processes that share VM (created with clone(CLONE_VM)). 
> 
> made a patch that adds in /proc/<pid>/status a VmClones field that tells
> how many proc that uses the same VM (mm_struct).  if there are clones I
> add another field VmFirstClone with the pid of clone with the lowest
> pid. 
> 
> Needed for things like gtop that adds mem usage of groups of proc, or
> else they add up the usage of SIZE and RSS of threads.
> 
> The patch need to be applied to linux/fs/proc/array.c

The basic idea is a good one (I have written a similar thing in the past ;)
Your implementation is O(n^2) however in ps, which is not acceptable.
Much better is it to add a new field to mm_struct that gets initialised
on first creation with the pid, and adding a place holder in pid hash
if that process goes away and the mm_struct is still there to avoid pid
reuse (or alternatively link task_structs to mms and always use the pid of
the first entry)

-Andi

Re: [PATCH] processes with shared vm

[Robert Love]

On 17 Aug 2001 10:21:35 +0200, Andi Kleen wrote:
> The basic idea is a good one (I have written a similar thing in the past ;)
> Your implementation is O(n^2) however in ps, which is not acceptable.
> <snip>

Is there any reason your patch was not accepted?  Perhaps for 2.5?

This is something (along with userspace changes to take advtantage of
it) that I think is really needed -- no more bogus ps/top reports.

I liked Terje's idea, but obviously the scalability needs to be improved
(I didn't even notice it, sadly).  I would really want to see this at
some point.

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] processes with shared vm

[Terje Eggestad]

Den 17 Aug 2001 10:21:35 +0200, skrev Andi Kleen:
> Terje Eggestad <terje.eggestad@scali.no> writes:
> 
> > I figured out that it's difficult to find out from /proc
> > which processes that share VM (created with clone(CLONE_VM)). 
> > 
> > made a patch that adds in /proc/<pid>/status a VmClones field that tells
> > how many proc that uses the same VM (mm_struct).  if there are clones I
> > add another field VmFirstClone with the pid of clone with the lowest
> > pid. 
> > 
> > Needed for things like gtop that adds mem usage of groups of proc, or
> > else they add up the usage of SIZE and RSS of threads.
> > 
> > The patch need to be applied to linux/fs/proc/array.c
> 
> The basic idea is a good one (I have written a similar thing in the past ;)
> Your implementation is O(n^2) however in ps, which is not acceptable.
> Much better is it to add a new field to mm_struct that gets initialised
> on first creation with the pid, and adding a place holder in pid hash
> if that process goes away and the mm_struct is still there to avoid pid
> reuse (or alternatively link task_structs to mms and always use the pid of
> the first entry)
> 
> -Andi

Thought of all that, yes ps will have O(n^2) BUT ONLY FOR CLONED PROCS.
How many cloned procs do you usually have????

Even if I agree that there should be a linked list of all the cloned
procs, it means major changes to the data structs in the kernel.

With the number of threaded programs out there, this is "good enough".

TJ


-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

Re: [PATCH] processes with shared vm

[Dave McCracken]

--On Friday, August 17, 2001 10:46:59 +0200 Terje Eggestad 
<terje.eggestad@scali.no> wrote:

> hought of all that, yes ps will have O(n^2) BUT ONLY FOR CLONED PROCS.
> How many cloned procs do you usually have????
>
> Even if I agree that there should be a linked list of all the cloned
> procs, it means major changes to the data structs in the kernel.
>
> With the number of threaded programs out there, this is "good enough".

There is a simpler way to do this.  All tasks belong to a thread group, and 
while thread groups are connected via a different clone flag 
(CLONE_THREAD), in practice CLONE_THREAD and CLONE_VM are generally used 
together.  It would be trivial to add TGID to the information in /proc, 
then assume all tasks with the same TGID have the same VM as well.  It 
would be just one more line in the /proc output and not require any 
additional overhead.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [PATCH] processes with shared vm

[Andi Kleen]

On Fri, Aug 17, 2001 at 09:26:10AM -0500, Dave McCracken wrote:
> There is a simpler way to do this.  All tasks belong to a thread group, and 
> while thread groups are connected via a different clone flag 
> (CLONE_THREAD), in practice CLONE_THREAD and CLONE_VM are generally used 
> together.  It would be trivial to add TGID to the information in /proc, 
> then assume all tasks with the same TGID have the same VM as well.  It 
> would be just one more line in the /proc output and not require any 
> additional overhead.

Even with a tgid you would need some way to avoid its counter wrapping
and getting reused. I think reusing the pid and pid hash for that is 
cheaper. Also gtop should display correct results even with the programs
that don't use CLONE_THREAD.

-Andi

Re: [PATCH] processes with shared vm

[Dave McCracken]

--On Friday, August 17, 2001 22:55:37 +0200 Andi Kleen <ak@suse.de> wrote:

> Even with a tgid you would need some way to avoid its counter wrapping
> and getting reused.

While in theory the pid that is used for tgid should never die while the 
thread group exists, this case needs to be handled for thread groups in 
general.  The number shouldn't be re-used for a pid as long as it's in use 
as a tgid.

> Also gtop should display correct results even with the programs
> that don't use CLONE_THREAD.

Are there any programs that use CLONE_VM and not CLONE_THREAD?

Actually I think we should make tgid visible in /proc in general because 
it's a useful thing to know, whether it's the right mechanism for gtop or 
not.  I'll work up a patch.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [PATCH] processes with shared vm

[Terje Eggestad]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2443 bytes --]

On Fri, 17 Aug 2001, Dave McCracken wrote:

> --On Friday, August 17, 2001 22:55:37 +0200 Andi Kleen <ak@suse.de> wrote:
>
> > Even with a tgid you would need some way to avoid its counter wrapping
> > and getting reused.
>
> While in theory the pid that is used for tgid should never die while the
> thread group exists, this case needs to be handled for thread groups in
> general.  The number shouldn't be re-used for a pid as long as it's in use
> as a tgid.

Be careful, I tested by using _clone() dircetly and let a proc A
create two clones B and C.  I then let A die, and B &C's ppid was now 1.
Attaching this code, you may find it useful for your tgid patch.
(this code is pretty unclean, ran into the problem that if A cloned B
and B cloned C, and now B died B remained a zombie and I didn't seem to
be able to catch SIGCHLD in A. (wanted to se C's ppid go to 1), probably
something stupid, just didn't bother to pursue it....)

THeory is correct if every one is using pthreads...
(exactly how many thread libs are ut there.....??)

>
> > Also gtop should display correct results even with the programs
> > that don't use CLONE_THREAD.
>
> Are there any programs that use CLONE_VM and not CLONE_THREAD?

attached program ;-)
Maybe not, but *the kernel allows it*!

>
> Actually I think we should make tgid visible in /proc in general because
> it's a useful thing to know, whether it's the right mechanism for gtop or
> not.  I'll work up a patch.

Looking forward to it!

>
> Dave McCracken
>
> ======================================================================
> Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
> dmccr@us.ibm.com                                        T/L   678-3059
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>



TJ

-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY
_________________________________________________________________________

[-- Attachment #2: Type: TEXT/PLAIN, Size: 1479 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <signal.h>
#include <sched.h>


pid_t pid0, pid1, pid2;
char * stack1, * stack2;

#define STACKSIZE 1024*8*4

struct sigcall{
  int sig;
  pid_t pid;
} siglist[1024];
int nsig = 0;

printfsiglist()
{
  int i;
  for (i = 0; i< nsig; i++) 
    printf("%d:: %d: %d %d\n", getpid(), i, siglist[nsig].pid, siglist[nsig].sig);
};

void sighand(int sig)
{
  siglist[nsig].pid = getpid();
  siglist[nsig].sig = sig;
  nsig++;
}

int clone2()
{

  pid2 = getpid();
  printf("clone2 has pid %d\n", pid2);

  sleep(100);
  printfsiglist();
};


int clone1()
{
  char * stack;

  pid1 = getpid();
  printf("clone1 has pid %d\n", pid1);

  sleep(10);
  printfsiglist();
  return;
  exit(0);
};

main()
{
  char * stack;
  int x, rc;
  pid0 = getpid();
  printf("clone0 has pid %d\n", pid0);

  signal(SIGCHLD, sighand);
  signal(SIGALRM, sighand);
  stack1 = malloc(STACKSIZE);
  stack2 = malloc(STACKSIZE);
  stack1 += STACKSIZE;
  stack2 += STACKSIZE;
  pid1 = __clone(clone1, stack1, CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND, NULL);
  pid2 = __clone(clone2, stack2, CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND, NULL);

  sleep(1);
  printf("master: clone0 has pid %d\n", pid0);
  printf("master: clone1 has pid %d\n", pid1);
  printf("master: clone2 has pid %d\n", pid2);

  exit(2);


};
  

Re: [PATCH] processes with shared vm

[Mike Touloumtzis]

On Fri, Aug 17, 2001 at 04:15:50PM -0500, Dave McCracken wrote:
> --On Friday, August 17, 2001 22:55:37 +0200 Andi Kleen <ak@suse.de> wrote:
> 
> >Also gtop should display correct results even with the programs
> >that don't use CLONE_THREAD.
> 
> Are there any programs that use CLONE_VM and not CLONE_THREAD?

Yes, I have written an emulation layer for an embedded OS which
does its own preemptive scheduling using signals.  It uses CLONE_VM
but not CLONE_THREAD or CLONE_SIGHAND.  Currently it is very
painful to debug with gdb :-(, although the multithreaded core
dumps in -ac are a godsend.

miket

[PATCH] processes with shared vm

[Terje Eggestad]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

I figured out that it's difficult to find out from /proc
which processes that share VM (created with clone(CLONE_VM)). 

made a patch that adds in /proc/<pid>/status a VmClones field that tells
how many proc that uses the same VM (mm_struct).  if there are clones I
add another field VmFirstClone with the pid of clone with the lowest
pid. 

Needed for things like gtop that adds mem usage of groups of proc, or
else they add up the usage of SIZE and RSS of threads.

The patch need to be applied to linux/fs/proc/array.c

NB: developed on 2.4.3

TJ

-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

[-- Attachment #2: vmclone.patch --]
[-- Type: text/x-patch, Size: 2250 bytes --]

*** array.c	Thu Aug 16 16:33:56 2001
--- array.c.orig	Mon Mar 19 21:34:55 2001
***************
*** 50,61 ****
   * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
   *			 :  proc_misc.c. The rest may eventually go into
   *			 :  base.c too.
-  *
-  * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
-  *                   :  that tells how many proc that uses the same VM (mm_struct).
-  *                   :  if there are clones add another field VmFirstClone with the
-  *                   :  clone with the lowest pid. Needed for things like gtop that adds 
-  *                   :  mem usage of groups of proc, or else they add up the usage of threads.
   */
  
  #include <linux/config.h>
--- 50,55 ----
***************
*** 184,190 ****
  static inline char * task_mem(struct mm_struct *mm, char *buffer)
  {
  	struct vm_area_struct * vma;
!  	unsigned long data = 0, stack = 0;
  	unsigned long exec = 0, lib = 0;
  
  	down_read(&mm->mmap_sem);
--- 178,184 ----
  static inline char * task_mem(struct mm_struct *mm, char *buffer)
  {
  	struct vm_area_struct * vma;
! 	unsigned long data = 0, stack = 0;
  	unsigned long exec = 0, lib = 0;
  
  	down_read(&mm->mmap_sem);
***************
*** 212,235 ****
  		"VmData:\t%8lu kB\n"
  		"VmStk:\t%8lu kB\n"
  		"VmExe:\t%8lu kB\n"
! 		"VmLib:\t%8lu kB\n"
! 		"VmClones:\t%d\n",
  		mm->total_vm << (PAGE_SHIFT-10),
  		mm->locked_vm << (PAGE_SHIFT-10),
  		mm->rss << (PAGE_SHIFT-10),
  		data - stack, stack,
! 		exec - lib, lib, 
! 		mm->mm_users.counter-2);
! 	/* if we've vm clones, find the lowest/first pid of the clones */	
! 	if (mm->mm_users.counter > 2) {
! 	  struct task_struct *p;
! 	  read_lock(&tasklist_lock);
! 	  for_each_task(p) {
! 	    if (p->mm == mm) break;
! 	  };
! 	  buffer += sprintf(buffer, "VmFirstClone:\t%d\n", p->pid);
! 	  read_unlock(&tasklist_lock);
! 	};
  	up_read(&mm->mmap_sem);
  	return buffer;
  }
--- 206,217 ----
  		"VmData:\t%8lu kB\n"
  		"VmStk:\t%8lu kB\n"
  		"VmExe:\t%8lu kB\n"
! 		"VmLib:\t%8lu kB\n",
  		mm->total_vm << (PAGE_SHIFT-10),
  		mm->locked_vm << (PAGE_SHIFT-10),
  		mm->rss << (PAGE_SHIFT-10),
  		data - stack, stack,
! 		exec - lib, lib);
  	up_read(&mm->mmap_sem);
  	return buffer;
  }

Re: [PATCH] processes with shared vm

[Robert Love]

On 17 Aug 2001 09:50:06 +0200, Terje Eggestad wrote:
> I figured out that it's difficult to find out from /proc
> which processes that share VM (created with clone(CLONE_VM)). 

good idea, but use diff -u

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] processes with shared vm

[Terje Eggestad]

[-- Attachment #1: Type: text/plain, Size: 2813 bytes --]

OK

Den 17 Aug 2001 03:56:49 -0400, skrev Robert Love:
> On 17 Aug 2001 09:50:06 +0200, Terje Eggestad wrote:
> > I figured out that it's difficult to find out from /proc
> > which processes that share VM (created with clone(CLONE_VM)). 
> 
> good idea, but use diff -u
> 
> -- 
> Robert M. Love
> rml at ufl.edu
> rml at tech9.net


--- array.c     Thu Aug 16 16:33:56 2001
+++ array.c.orig        Mon Mar 19 21:34:55 2001
@@ -50,12 +50,6 @@
  * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
  *                      :  proc_misc.c. The rest may eventually go into
  *                      :  base.c too.
- *
- * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
- *                   :  that tells how many proc that uses the same VM
(mm_struct).
- *                   :  if there are clones add another field
VmFirstClone with the
- *                   :  clone with the lowest pid. Needed for things
like gtop that adds 
- *                   :  mem usage of groups of proc, or else they add
up the usage of threads.
  */
 
 #include <linux/config.h>
@@ -184,7 +178,7 @@
 static inline char * task_mem(struct mm_struct *mm, char *buffer)
 {
        struct vm_area_struct * vma;
-       unsigned long data = 0, stack = 0;
+       unsigned long data = 0, stack = 0;
        unsigned long exec = 0, lib = 0;
 
        down_read(&mm->mmap_sem);
@@ -212,24 +206,12 @@
                "VmData:\t%8lu kB\n"
                "VmStk:\t%8lu kB\n"
                "VmExe:\t%8lu kB\n"
-               "VmLib:\t%8lu kB\n"
-               "VmClones:\t%d\n",
+               "VmLib:\t%8lu kB\n",
                mm->total_vm << (PAGE_SHIFT-10),
                mm->locked_vm << (PAGE_SHIFT-10),
                mm->rss << (PAGE_SHIFT-10),
                data - stack, stack,
-               exec - lib, lib, 
-               mm->mm_users.counter-2);
-       /* if we've vm clones, find the lowest/first pid of the clones
*/
-       if (mm->mm_users.counter > 2) {
-         struct task_struct *p;
-         read_lock(&tasklist_lock);
-         for_each_task(p) {
-           if (p->mm == mm) break;
-         };
-         buffer += sprintf(buffer, "VmFirstClone:\t%d\n", p->pid);
-         read_unlock(&tasklist_lock);
-       };
+               exec - lib, lib);
        up_read(&mm->mmap_sem);
        return buffer;
 }


-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

[-- Attachment #2: vmclone.patch --]
[-- Type: text/plain, Size: 1693 bytes --]

--- array.c	Thu Aug 16 16:33:56 2001
+++ array.c.orig	Mon Mar 19 21:34:55 2001
@@ -50,12 +50,6 @@
  * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
  *			 :  proc_misc.c. The rest may eventually go into
  *			 :  base.c too.
- *
- * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
- *                   :  that tells how many proc that uses the same VM (mm_struct).
- *                   :  if there are clones add another field VmFirstClone with the
- *                   :  clone with the lowest pid. Needed for things like gtop that adds 
- *                   :  mem usage of groups of proc, or else they add up the usage of threads.
  */
 
 #include <linux/config.h>
@@ -184,7 +178,7 @@
 static inline char * task_mem(struct mm_struct *mm, char *buffer)
 {
 	struct vm_area_struct * vma;
- 	unsigned long data = 0, stack = 0;
+	unsigned long data = 0, stack = 0;
 	unsigned long exec = 0, lib = 0;
 
 	down_read(&mm->mmap_sem);
@@ -212,24 +206,12 @@
 		"VmData:\t%8lu kB\n"
 		"VmStk:\t%8lu kB\n"
 		"VmExe:\t%8lu kB\n"
-		"VmLib:\t%8lu kB\n"
-		"VmClones:\t%d\n",
+		"VmLib:\t%8lu kB\n",
 		mm->total_vm << (PAGE_SHIFT-10),
 		mm->locked_vm << (PAGE_SHIFT-10),
 		mm->rss << (PAGE_SHIFT-10),
 		data - stack, stack,
-		exec - lib, lib, 
-		mm->mm_users.counter-2);
-	/* if we've vm clones, find the lowest/first pid of the clones */	
-	if (mm->mm_users.counter > 2) {
-	  struct task_struct *p;
-	  read_lock(&tasklist_lock);
-	  for_each_task(p) {
-	    if (p->mm == mm) break;
-	  };
-	  buffer += sprintf(buffer, "VmFirstClone:\t%d\n", p->pid);
-	  read_unlock(&tasklist_lock);
-	};
+		exec - lib, lib);
 	up_read(&mm->mmap_sem);
 	return buffer;
 }

Re: [PATCH] processes with shared vm

[Robert Love]

On 17 Aug 2001 10:04:04 +0200, Terje Eggestad wrote:
> OK

the patch is backwards :) i think your original one was too

diff -u <old> <new>

-- 
Robert M. Love
rml at ufl.edu
rml at tech9.net

Re: [PATCH] processes with shared vm

[Terje Eggestad]

[-- Attachment #1: Type: text/plain, Size: 2782 bytes --]

Man......

Is there any other way I can do this wrong!?!?, guess I need to write
more patches :-)

--- array.c.orig	Mon Mar 19 21:34:55 2001
+++ array.c	Thu Aug 16 16:33:56 2001
@@ -50,6 +50,12 @@
  * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
  *			 :  proc_misc.c. The rest may eventually go into
  *			 :  base.c too.
+ *
+ * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
+ *                   :  that tells how many proc that uses the same VM (mm_struct).
+ *                   :  if there are clones add another field VmFirstClone with the
+ *                   :  clone with the lowest pid. Needed for things like gtop that adds 
+ *                   :  mem usage of groups of proc, or else they add up the usage of threads.
  */
 
 #include <linux/config.h>
@@ -178,7 +184,7 @@
 static inline char * task_mem(struct mm_struct *mm, char *buffer)
 {
 	struct vm_area_struct * vma;
-	unsigned long data = 0, stack = 0;
+ 	unsigned long data = 0, stack = 0;
 	unsigned long exec = 0, lib = 0;
 
 	down_read(&mm->mmap_sem);
@@ -206,12 +212,24 @@
 		"VmData:\t%8lu kB\n"
 		"VmStk:\t%8lu kB\n"
 		"VmExe:\t%8lu kB\n"
-		"VmLib:\t%8lu kB\n",
+		"VmLib:\t%8lu kB\n"
+		"VmClones:\t%d\n",
 		mm->total_vm << (PAGE_SHIFT-10),
 		mm->locked_vm << (PAGE_SHIFT-10),
 		mm->rss << (PAGE_SHIFT-10),
 		data - stack, stack,
-		exec - lib, lib);
+		exec - lib, lib, 
+		mm->mm_users.counter-2);
+	/* if we've vm clones, find the lowest/first pid of the clones */	
+	if (mm->mm_users.counter > 2) {
+	  struct task_struct *p;
+	  read_lock(&tasklist_lock);
+	  for_each_task(p) {
+	    if (p->mm == mm) break;
+	  };
+	  buffer += sprintf(buffer, "VmFirstClone:\t%d\n", p->pid);
+	  read_unlock(&tasklist_lock);
+	};
 	up_read(&mm->mmap_sem);
 	return buffer;
 }
Den 17 Aug 2001 04:08:53 -0400, skrev Robert Love:
> On 17 Aug 2001 10:04:04 +0200, Terje Eggestad wrote:
> > OK
> 
> the patch is backwards :) i think your original one was too
> 
> diff -u <old> <new>
> 
> -- 
> Robert M. Love
> rml at ufl.edu
> rml at tech9.net
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

[-- Attachment #2: vmclone.patch --]
[-- Type: text/plain, Size: 1693 bytes --]

--- array.c.orig	Mon Mar 19 21:34:55 2001
+++ array.c	Thu Aug 16 16:33:56 2001
@@ -50,6 +50,12 @@
  * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
  *			 :  proc_misc.c. The rest may eventually go into
  *			 :  base.c too.
+ *
+ * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
+ *                   :  that tells how many proc that uses the same VM (mm_struct).
+ *                   :  if there are clones add another field VmFirstClone with the
+ *                   :  clone with the lowest pid. Needed for things like gtop that adds 
+ *                   :  mem usage of groups of proc, or else they add up the usage of threads.
  */
 
 #include <linux/config.h>
@@ -178,7 +184,7 @@
 static inline char * task_mem(struct mm_struct *mm, char *buffer)
 {
 	struct vm_area_struct * vma;
-	unsigned long data = 0, stack = 0;
+ 	unsigned long data = 0, stack = 0;
 	unsigned long exec = 0, lib = 0;
 
 	down_read(&mm->mmap_sem);
@@ -206,12 +212,24 @@
 		"VmData:\t%8lu kB\n"
 		"VmStk:\t%8lu kB\n"
 		"VmExe:\t%8lu kB\n"
-		"VmLib:\t%8lu kB\n",
+		"VmLib:\t%8lu kB\n"
+		"VmClones:\t%d\n",
 		mm->total_vm << (PAGE_SHIFT-10),
 		mm->locked_vm << (PAGE_SHIFT-10),
 		mm->rss << (PAGE_SHIFT-10),
 		data - stack, stack,
-		exec - lib, lib);
+		exec - lib, lib, 
+		mm->mm_users.counter-2);
+	/* if we've vm clones, find the lowest/first pid of the clones */	
+	if (mm->mm_users.counter > 2) {
+	  struct task_struct *p;
+	  read_lock(&tasklist_lock);
+	  for_each_task(p) {
+	    if (p->mm == mm) break;
+	  };
+	  buffer += sprintf(buffer, "VmFirstClone:\t%d\n", p->pid);
+	  read_unlock(&tasklist_lock);
+	};
 	up_read(&mm->mmap_sem);
 	return buffer;
 }

Re: [PATCH] processes with shared vm

[michael]

Why not just print out the address of the 'mm_struct'? 

That lets 'ps' treat the address as a cookie, and
thus count the number of occurences of each vm.

This has the additional advantages of:

* moving the intelligence out to the app
* reducing the kernel size, and
* make it easy to find out exactly which processes
are using which vm. (you just search for all occurences
of the cookie).

Michael.


Terje Eggestad <terje.eggestad@scali.no> writes:
> --=-L6iLOYILsDzljNLIi035
> Content-Type: text/plain
> Content-Transfer-Encoding: 7bit
> --- array.c.orig	Mon Mar 19 21:34:55 2001
> +++ array.c	Thu Aug 16 16:33:56 2001
> @@ -50,6 +50,12 @@
>   * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
>   *			 :  proc_misc.c. The rest may eventually go into
>   *			 :  base.c too.
> + *
> + * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
> + *                   :  that tells how many proc that uses the same VM (mm_struct).
> + *                   :  if there are clones add another field VmFirstClone with the
> + *                   :  clone with the lowest pid. Needed for things like gtop that adds 
> + *                   :  mem usage of groups of proc, or else they add up the usage of threads.
>   */

Re: [PATCH] processes with shared vm

[Terje Eggestad]

On 17 Aug 2001 michael@optusnet.com.au wrote:

>
>
> Why not just print out the address of the 'mm_struct'?
>
> That lets 'ps' treat the address as a cookie, and
> thus count the number of occurences of each vm.
>
> This has the additional advantages of:
>
> * moving the intelligence out to the app
> * reducing the kernel size, and
> * make it easy to find out exactly which processes
> are using which vm. (you just search for all occurences
> of the cookie).
>
> Michael.
>

Not a bad idea, One reason is that I've an inate distrust of using
addresses as anything remotely useful. BTW, do you want the tag in hex or
dec??
keep in mind 32 and  64 bit machines, it must actually be a 64 bit!

What I really wanted was a list of pids of the clones.
ps/top/gtop could then use it as an exclude list for futher processing...
Trouble is that returning more than 4kb in a file in proc is a pain,
and there is no guarantee that someone make will not a 1000 clones.
ref the recent problem with maps exceeding 4kb.

(I might be paranoid, you get ~170 (1024/6) pids in 1kb, assuming 16bit
pid.

using the lowest pid seems a good compromise.

I still think the overhead is neglible.
What's the upper practial limit of procs ~64k? (more like 4k.)
How many instructions to tranvers the task list and test mm_struct
pointer for equality? O(10) per task.? assuming all clones we're talking
about ~650k instructions, and with 100mips machines (with 64k task, that's
slow1) thats 1/200 second overhead every time you do cat
/proc/[0-9]*/status. I can live with that.


But I guess I can live with using the address.....

>
> Terje Eggestad <terje.eggestad@scali.no> writes:
> > --=-L6iLOYILsDzljNLIi035
> > Content-Type: text/plain
> > Content-Transfer-Encoding: 7bit
> > --- array.c.orig	Mon Mar 19 21:34:55 2001
> > +++ array.c	Thu Aug 16 16:33:56 2001
> > @@ -50,6 +50,12 @@
> >   * Al Viro & Jeff Garzik :  moved most of the thing into base.c and
> >   *			 :  proc_misc.c. The rest may eventually go into
> >   *			 :  base.c too.
> > + *
> > + * Terje Eggestad    :  added in /proc/<pid>/status a VmClones: n
> > + *                   :  that tells how many proc that uses the same VM (mm_struct).
> > + *                   :  if there are clones add another field VmFirstClone with the
> > + *                   :  clone with the lowest pid. Needed for things like gtop that adds
> > + *                   :  mem usage of groups of proc, or else they add up the usage of threads.
> >   */
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY
_________________________________________________________________________

Re: [PATCH] processes with shared vm

[Albert D. Cahalan]

Terje Eggestad writes:
> On 17 Aug 2001 michael@optusnet.com.au wrote:

>> Why not just print out the address of the 'mm_struct'?
>>
>> That lets 'ps' treat the address as a cookie, and
>> thus count the number of occurences of each vm.

No, I won't make 'ps' do that. Ever wonder why 'ps' doesn't
sort processes by default? It isn't OK to suck up lots of
memory or reparse the files. This is bad for performance and
causes extra trouble when a kernel bug causes /proc/42/status to
freeze the 'ps' process.

Also your proposal would require 'ps' to _always_ read the
data for _every_ process in the system.

> Not a bad idea, One reason is that I've an inate distrust of using
> addresses as anything remotely useful. BTW, do you want the tag in hex
> or dec??

Either... hex is nice I guess.

> keep in mind 32 and  64 bit machines, it must actually be a 64 bit!

Nah, this gets you enough:   (unsigned)(ptr_to_mm>>sizeof(long))

> What I really wanted was a list of pids of the clones.
> ps/top/gtop could then use it as an exclude list for futher processing...

Nope. This is not enough for sane thread support in 'ps'.
Information is lost across the kernel-user boundry. It would
be relatively easy for the kernel to provide a /proc directory
listing that groups processes by mm_struct. Without this, 'ps'
would have to regenerate the lost information in an inefficient
way.

BTW, 'ps' is now here:  http://procps.sourceforge.net/

> Trouble is that returning more than 4kb in a file in proc is a pain,
> and there is no guarantee that someone make will not a 1000 clones.
> ref the recent problem with maps exceeding 4kb.
> 
> (I might be paranoid, you get ~170 (1024/6) pids in 1kb, assuming 16bit
> pid.

Yes, one has to assume that some cracker or sicko will do that.

> using the lowest pid seems a good compromise.

The PID may wrap.

> I still think the overhead is neglible.
> What's the upper practial limit of procs ~64k? (more like 4k.)

On an SSI cluster, way more I'd guess.

> How many instructions to tranvers the task list and test mm_struct
> pointer for equality? O(10) per task.? assuming all clones we're talking
> about ~650k instructions, and with 100mips machines (with 64k task, that's
> slow1) thats 1/200 second overhead every time you do cat
> /proc/[0-9]*/status. I can live with that.

You also dirty the cache and suffer load misses.

Re: [PATCH] processes with shared vm

[Terje Eggestad]

On Sun, 19 Aug 2001, Albert D. Cahalan wrote:

> Terje Eggestad writes:
> > On 17 Aug 2001 michael@optusnet.com.au wrote:
>
> >> Why not just print out the address of the 'mm_struct'?
> >>
> >> That lets 'ps' treat the address as a cookie, and
> >> thus count the number of occurences of each vm.
>
> No, I won't make 'ps' do that. Ever wonder why 'ps' doesn't
> sort processes by default? It isn't OK to suck up lots of
> memory or reparse the files. This is bad for performance and
> causes extra trouble when a kernel bug causes /proc/42/status to
> freeze the 'ps' process.
>
> Also your proposal would require 'ps' to _always_ read the
> data for _every_ process in the system.
>

BTW, Why is ps reading the status file at all????
what info is there that is not in cmndline/stat/statm??



> > Not a bad idea, One reason is that I've an inate distrust of using
> > addresses as anything remotely useful. BTW, do you want the tag in hex
> > or dec??
>
> Either... hex is nice I guess.
>
> > keep in mind 32 and  64 bit machines, it must actually be a 64 bit!
>
> Nah, this gets you enough:   (unsigned)(ptr_to_mm>>sizeof(long))
>

hmmmmm, the MSB 32 bit??? that would almost always be the same for all
kernel pointers, surely you mean (unsigned) (ptr_to_mm & 0xffffffff) ??

A point though, guess the kernel is never going to use > 4Gb of VM.

> > What I really wanted was a list of pids of the clones.
> > ps/top/gtop could then use it as an exclude list for futher processing...
>
> Nope. This is not enough for sane thread support in 'ps'.
> Information is lost across the kernel-user boundry. It would
> be relatively easy for the kernel to provide a /proc directory
> listing that groups processes by mm_struct. Without this, 'ps'
> would have to regenerate the lost information in an inefficient
> way.

Not sure what you're getting at if you use the address of mm_struct as
a clone tag,  you *do* have to keep track of all read clone tags, right?

What DO you need for sane thread support.....

> BTW, 'ps' is now here:  http://procps.sourceforge.net/
>
> > Trouble is that returning more than 4kb in a file in proc is a pain,
> > and there is no guarantee that someone make will not a 1000 clones.
> > ref the recent problem with maps exceeding 4kb.
> >
> > (I might be paranoid, you get ~170 (1024/6) pids in 1kb, assuming 16bit
> > pid.
>
> Yes, one has to assume that some cracker or sicko will do that.
>
> > using the lowest pid seems a good compromise.
>
> The PID may wrap.
>

kept thinking that the first clone is unlikely to ever die, but
you're right, new clones may very well have a lower pid, which will
of course throw of ps and likes. (damn)


> > I still think the overhead is neglible.
> > What's the upper practial limit of procs ~64k? (more like 4k.)
>
> On an SSI cluster, way more I'd guess.
>
> > How many instructions to tranvers the task list and test mm_struct
> > pointer for equality? O(10) per task.? assuming all clones we're talking
> > about ~650k instructions, and with 100mips machines (with 64k task, that's
> > slow1) thats 1/200 second overhead every time you do cat
> > /proc/[0-9]*/status. I can live with that.
>
> You also dirty the cache and suffer load misses.

hmmm, put it like that I also prevent future cache misses, ps is likely to
travers the entire task_struct list anyway...
But, sure, fine, 1/20 of a second, prefetching algo's is getting pretty
good.
64k tasks....  aren't we both getting slightly farfetched here...

TJ


-- 
_________________________________________________________________________

Terje Eggestad                  terje.eggestad@scali.no
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY
_________________________________________________________________________

Re: [PATCH] processes with shared vm

[Albert D. Cahalan]

Terje Eggestad writes:
> On Sun, 19 Aug 2001, Albert D. Cahalan wrote:
>> Terje Eggestad writes:
>>> On 17 Aug 2001 michael@optusnet.com.au wrote:

>>>> Why not just print out the address of the 'mm_struct'?
>>>>
>>>> That lets 'ps' treat the address as a cookie, and
>>>> thus count the number of occurences of each vm.
>>
>> No, I won't make 'ps' do that. Ever wonder why 'ps' doesn't
>> sort processes by default? It isn't OK to suck up lots of
>> memory or reparse the files. This is bad for performance and
>> causes extra trouble when a kernel bug causes /proc/42/status to
>> freeze the 'ps' process.
>>
>> Also your proposal would require 'ps' to _always_ read the
>> data for _every_ process in the system.
>
> BTW, Why is ps reading the status file at all????
> what info is there that is not in cmndline/stat/statm??

all 4 UIDs
all 64 signals
misc. VM stats (maybe)

>>> Not a bad idea, One reason is that I've an inate distrust of using
>>> addresses as anything remotely useful. BTW, do you want the tag in hex
>>> or dec??
>>
>> Either... hex is nice I guess.
>>
>>> keep in mind 32 and  64 bit machines, it must actually be a 64 bit!
>>
>> Nah, this gets you enough:   (unsigned)(ptr_to_mm>>sizeof(long))
>
> hmmmmm, the MSB 32 bit??? that would almost always be the same for all
> kernel pointers, surely you mean (unsigned) (ptr_to_mm & 0xffffffff) ??
>
> A point though, guess the kernel is never going to use > 4Gb of VM.

sizeof(long) is 4 or 8

the struct is several hundred bytes on 64-bit hardware

shifting by 8 would allow for a terabyte of kernel memory

>>> What I really wanted was a list of pids of the clones.
>>> ps/top/gtop could then use it as an exclude list for futher processing...
>>
>> Nope. This is not enough for sane thread support in 'ps'.
>> Information is lost across the kernel-user boundry. It would
>> be relatively easy for the kernel to provide a /proc directory
>> listing that groups processes by mm_struct. Without this, 'ps'
>> would have to regenerate the lost information in an inefficient
>> way.
>
> Not sure what you're getting at if you use the address of mm_struct
> as a clone tag,  you *do* have to keep track of all read clone tags,
> right?

There wouldn't be a need to keep track of all read clone tags if the
kernel would group processes by clone tag. Then 'ps' would only need
to remember the current clone tag and the accumulated info.

> What DO you need for sane thread support.....

Avoiding the "thread" and "process" terms here:

The above would do. When 'ps' looks at a task, 'ps' must be able to
determine if the task should be grouped with the previous or next one.
That, and the partial ordering implied, would be enough to collect
tasks into groups. I don't expect much trouble selecting the leader,
since I have the "PPid" (parent TASK identifier!) value.

>>> I still think the overhead is neglible.
>>> What's the upper practial limit of procs ~64k? (more like 4k.)
>>
>> On an SSI cluster, way more I'd guess.
>>
>>> How many instructions to tranvers the task list and test mm_struct
>>> pointer for equality? O(10) per task.? assuming all clones we're talking
>>> about ~650k instructions, and with 100mips machines (with 64k task, that's
>>> slow1) thats 1/200 second overhead every time you do cat
>>> /proc/[0-9]*/status. I can live with that.
>>
>> You also dirty the cache and suffer load misses.
>
> hmmm, put it like that I also prevent future cache misses, ps is likely to
> travers the entire task_struct list anyway...
> But, sure, fine, 1/20 of a second, prefetching algo's is getting pretty
> good.
> 64k tasks....  aren't we both getting slightly farfetched here...

I don't know. Ask on the linux-cluster list.