Thread scheduling on an SMP

Thread scheduling on an SMP

[Darío Mariani]

Hello:
  I have a machine with 4 processors, and I wold like to make the
following: Set 1 processor to do general stuff using the default
scheduler and the other three working on my program exclusively, with
the realtime or my own thread scheduler.
  Is this possible?, thanks for any comment or links to documentation
you can give me.

              Darío
-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Thread scheduling on an SMP

[Richard Nairn]

If I am reading your question correctly, you are talking about schedular  
affinity. You can set which processors you would like to assign pid's to.  
You need a kernel after 2.5.8 for thiese functions...

Excerpt:

#include <sched.h>

int sched_setaffinity(pid_t pid, unsigned int len, unsigned long *mask);

int sched_getaffinity(pid_t pid, unsigned int len, unsigned long *mask);

DESCRIPTION
sched_setaffinity sets the CPU affinity mask of the process denoted by  
pid. If pid is zero, then the current process is used.

The affinity mask is represented by the bitmask stored in mask. The least  
significant bit corresponds to the first logical processor number on the  
system, while the most significant bit corresponds to the last logical  
processor number on the system. A set bit corresponds to a legally  
schedulable CPU while an unset bit corresponds to an illegally schedulable  
CPU. In other words, a process is bound to and will only run on processors  
whose corresponding bit is set. Usually, all bits in the mask are set.

The argument len is the length of the data pointed to by mask. Normally  
this is the size of a word on the system. For compatibility with future  
versions of the Linux kernel, since this size can change, the bitmask  
supplied must be at least as large as the affinity mask stored in the  
kernel.

The function sched_getaffinity writes into the pointer supplied by mask  
that is size len the affinity mask of process pid. If pid is zero, then  
the mask of the current process is returned.

On Thu, 23 Dec 2004 10:14:17 -0300, Darío Mariani  
<mariani.dario@gmail.com> wrote:

> Hello:
>   I have a machine with 4 processors, and I wold like to make the
> following: Set 1 processor to do general stuff using the default
> scheduler and the other three working on my program exclusively, with
> the realtime or my own thread scheduler.
>   Is this possible?, thanks for any comment or links to documentation
> you can give me.
>
>               Darío
> -
> To unsubscribe from this list: send the line "unsubscribe  
> linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
  |       Richard Nairn          Specializing in Linux
  |     Nairn Consulting         Web / Database Solutions
  |        Calgary, AB
  | Richard@NairnConsulting.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

file deletion

[Andy]

Does anybody know of any useful code or c commands that
you can use to search for duplicate files within a linux/unix directory and
its subdirectories and remove them?
Thanks
Andrew

Re: file deletion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

On Fri, 2004-12-24 13:14:11 -0000, Andy <andy_webb@onetel.com>
wrote in message <003f01c4e9ba$7679d020$316c4ed5@j0s6l8>:
> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?

As long as you believe in cryptographical hashes, something like this
should do the trick:

SOME_DIR=/some/directory

find "${SOME_DIR}" -type f -exec sha1sum {} \; | \
	sort | \
	uniq -c | \
	sed -e 's/^[[:space:]]*//g' | \
	egrep -v '\<1\>' | \
	cut -f 4- -d ' ' | \
	while read DOUBLE_FILE_NAME; do
		rm -f "${SOME_DIR}/${DOUBLE_FILE_NAME}"
	done

That's untested, but should work (it's essentially a one-liner).
However, it removes *all* instances of files which are believed to be
identical...

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: file deletion

[J.]

On Fri, 24 Dec 2004, Andy wrote:

> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?
> Thanks
> Andrew

Ehm.. Personally no.. However there is ftw, opendir, readdir, stat,
fstat... etc.. Ones you are able to access all the directory entries you
have to make a decision how you want to compare e.g. md5, crc32, only size
or name etc... Then there is the issue of choosing a optimal ADT and
access/retrieval algo.

If you dont have to c code the program but just looking for a solution I
would most certainly go for a: 
`find -type f -exec md5sum '{}' \; >> md5.log` 
and parse the md5.log with a simple shell or awk script. That would save
many headache's.. Plus you don't have to reinvent `find`... 

J.

--
http://www.rdrs.net

Re: file deletion

[Glynn Clements]

Andy wrote:

> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?

First, you have to define "duplicate". Also, once you've decided that
two or more files are duplicates, you have to decide which one you
wish to keep and which ones are to be deleted.

If you consider any files with identical MD5 hashes as duplicates, and
don't care about which one is kept, you could use something like:

find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
	while read file ; do rm -- "$file" ; done

However, if you have 3 or more copies of a given file, the above will
only delete one of them ("uniq -d ..." only prints one instance of
each duplicate line; "uniq -D ..." prints all instances, which would
result in all copies being removed; there isn't an all-but-one option).

-- 
Glynn Clements <glynn@gclements.plus.com>

Re: file deletion

[wwp]

Hello Glynn et al,


On Thu, 30 Dec 2004 11:26:21 +0000 Glynn Clements <glynn@gclements.plus.com> wrote:

> 
> Andy wrote:
> 
> > Does anybody know of any useful code or c commands that
> > you can use to search for duplicate files within a linux/unix directory and
> > its subdirectories and remove them?
> 
> First, you have to define "duplicate". Also, once you've decided that
> two or more files are duplicates, you have to decide which one you
> wish to keep and which ones are to be deleted.
> 
> If you consider any files with identical MD5 hashes as duplicates, and
> don't care about which one is kept, you could use something like:
> 
> find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
> 	while read file ; do rm -- "$file" ; done
> 
> However, if you have 3 or more copies of a given file, the above will
> only delete one of them ("uniq -d ..." only prints one instance of
> each duplicate line; "uniq -D ..." prints all instances, which would
> result in all copies being removed; there isn't an all-but-one option).

An idea would also be to remove all-but-one and create hardlinks in place of
the removed ones, maybe? Of course it would depend Andy's answers to the
questions raised by Glynn: what means 'duplicate' and what to do w/ those
dups (even why are you checkinf for dups and what are you expecting to be
able to do?).


Regards,

-- 
wwp

Re: file deletion

[Andy]

Gents,
thanks for this, this script seems to do the trick.
Actually what I meant by "duplicate" but didn't clarify
was identical name and file size, dates do not matter.
What does md5sum {} do?
Thanks
Andy
----- Original Message -----
From: "wwp" <subscript@free.fr>
To: <linux-c-programming@vger.kernel.org>
Sent: Thursday, December 30, 2004 12:04 PM
Subject: Re: file deletion


> Hello Glynn et al,
>
>
> On Thu, 30 Dec 2004 11:26:21 +0000 Glynn Clements
<glynn@gclements.plus.com> wrote:
>
> >
> > Andy wrote:
> >
> > > Does anybody know of any useful code or c commands that
> > > you can use to search for duplicate files within a linux/unix
directory and
> > > its subdirectories and remove them?
> >
> > First, you have to define "duplicate". Also, once you've decided that
> > two or more files are duplicates, you have to decide which one you
> > wish to keep and which ones are to be deleted.
> >
> > If you consider any files with identical MD5 hashes as duplicates, and
> > don't care about which one is kept, you could use something like:
> >
> > find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
> > while read file ; do rm -- "$file" ; done
> >
> > However, if you have 3 or more copies of a given file, the above will
> > only delete one of them ("uniq -d ..." only prints one instance of
> > each duplicate line; "uniq -D ..." prints all instances, which would
> > result in all copies being removed; there isn't an all-but-one option).
>
> An idea would also be to remove all-but-one and create hardlinks in place
of
> the removed ones, maybe? Of course it would depend Andy's answers to the
> questions raised by Glynn: what means 'duplicate' and what to do w/ those
> dups (even why are you checkinf for dups and what are you expecting to be
> able to do?).
>
>
> Regards,
>
> --
> wwp
> -
> To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: file deletion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

On Sat, 2005-01-01 11:09:37 -0000, Andy <andy_webb@onetel.com>
wrote in message <000b01c4eff2$623ff7e0$8a7e4ed5@j0s6l8>:
> Gents,
> thanks for this, this script seems to do the trick.
> Actually what I meant by "duplicate" but didn't clarify
> was identical name and file size, dates do not matter.
> What does md5sum {} do?

The "find" command will substitute "{}" inside an "-exec ... ;"
expression with the filename it currently checks. So this is only a
md5sum call with a given filename.

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Linux source code: malloc.c

[Venkatesh Joshi]

Hi,

Where can I see the implementation of free() and malloc() in Linux
source code? I am not able to locate the file malloc.c.

thanks,
venkatesh

Re: Linux source code: malloc.c

[Steven Smith]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

> Where can I see the implementation of free() and malloc() in Linux
> source code? I am not able to locate the file malloc.c.
That's because malloc() isn't in Linux-the-kernel: it's in glibc.
The file you're looking for is malloc/malloc.c in the glibc tarball.

Steven Smith.
-- 
One day, I'm going to get an Alice-bot to answer all my email for me,
and see how long it takes people to notice.

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: Linux source code: malloc.c

[davidgn]

Mensaje citado por: Steven Smith <sos22@cam.ac.uk>:

> > Where can I see the implementation of free() and malloc() in Linux
> > source code? I am not able to locate the file malloc.c.
> That's because malloc() isn't in Linux-the-kernel: it's in glibc.
> The file you're looking for is malloc/malloc.c in the glibc tarball.
> 

Except the kernel implements its own "kmalloc", which if I'm not wrong is called
upon by libc's malloc


-------------------------------------------------
www.correo.unam.mx
UNAMonos Comunicándonos

-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux source code: malloc.c

[Steven Smith]

[-- Attachment #1: Type: text/plain, Size: 933 bytes --]

> > > Where can I see the implementation of free() and malloc() in Linux
> > > source code? I am not able to locate the file malloc.c.
> > That's because malloc() isn't in Linux-the-kernel: it's in glibc.
> > The file you're looking for is malloc/malloc.c in the glibc tarball.
> Except the kernel implements its own "kmalloc", which if I'm not wrong is called
> upon by libc's malloc
Um, sort of... the kernel does implement a function kmalloc, but it's
not the same as the user-exposed one.  For a start, memory allocated by
kmalloc is only accessible from kernel space, rather than user programs.

The kernel does provide two main primitives for userspace programs to
allocated memory: mmap and brk.  These are both fairly limited,
though.  mmap can only cope with multiples of the page size (4096
bytes on x86), while brk enforces a stack-like discipline.

sys_mmap and sys_brk do call upon kmalloc themselves, though.

Steven.


[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]