[RFC] POSIX personality

[RFC] POSIX personality

[Dave McCracken]

As part of improving support for POSIX multithreading I've been putting
together some patches to allow more things to be shared between tasks.
Right now this is accomplished via flags to clone() with one flag per
resource to be shared.  This usually translates to a data structure pointed
to out of task_struct, complete with reference count and lock.

In a discussion today an alternate idea was proposed by Ben LaHaise.  He
suggested creating a POSIX personality, or execution domain.  This would
take some pressure off the clone flag space as well as allowing some
optimizations in the code. It could also be used in situations where
POSIX-compatible behavior entails more than just sharing extra resources
between tasks.

This would assume that the resources I'm sharing would only be useful for
POSIX compatibility, but at this point it seems unlikely that anyone would
want to share a subset of them.  The resources I'm currently working on
include credentials, signals,  and timers, and there's a patch available
for semaphore undo that could also be part of this mechanism.

Since you've made it this far my question to you all is this:  assuming
that we do want improved POSIX compatibility does this sound like a
reasonable way to add it?

Thanks,
Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [RFC] POSIX personality

[Linus Torvalds]

On Tue, 21 May 2002, Dave McCracken wrote:
>
> In a discussion today an alternate idea was proposed by Ben LaHaise.  He
> suggested creating a POSIX personality, or execution domain.  This would
> take some pressure off the clone flag space as well as allowing some
> optimizations in the code. It could also be used in situations where
> POSIX-compatible behavior entails more than just sharing extra resources
> between tasks.

I don't see any reason to start using some fixed-mode semantics without 
seeing some stronger arguments on exactly why that would be a good idea. 
We have used up 11 of 24 bits (and more can be made available) over the 
last five years, and there are no obvious inefficiencies that I can see.

Also, stateful computing is usually a _bad_ idea. System calls that behave
differently depending on some external state should be avoided. Yes, it's
largely unavoidable if the external state is "we now emulate some other OS
behaviour", but that's not an excuse to create new such state, imnsho.

		Linus

Re: [RFC] POSIX personality

[Dave McCracken]

--On Tuesday, May 21, 2002 01:52:37 PM -0700 Linus Torvalds
<torvalds@transmeta.com> wrote:

> I don't see any reason to start using some fixed-mode semantics without 
> seeing some stronger arguments on exactly why that would be a good idea. 
> We have used up 11 of 24 bits (and more can be made available) over the 
> last five years, and there are no obvious inefficiencies that I can see.

Ok, sounds reasonable.  I'll add the bits as I go, then.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [RFC] POSIX personality

[Bill Davidsen]

On Tue, 21 May 2002, Linus Torvalds wrote:

> I don't see any reason to start using some fixed-mode semantics without 
> seeing some stronger arguments on exactly why that would be a good idea. 
> We have used up 11 of 24 bits (and more can be made available) over the 
> last five years, and there are no obvious inefficiencies that I can see.

I think the reason which comes to mind is avoiding future problems. By
having a single POSIX mode flag not only does the program not have to know
about setting the "right" other bits today, but if we find that POSIX
behaviour is needed in some other area in the future, the program doesn't
need to be modified and recompiled, because the POSIX behaviour "is in
there" for all things.

Ideally a program could execute some make-Linux-run-POSIX in init, and
have the expected behaviour everywhere without all manner of ifdefs.

I think that's a stronger argument, avoiding future problems is nice, like
wearing Kevlar sox if you shoot yourself in the foot a lot.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [RFC] POSIX personality

[Linus Torvalds]

On Thu, 23 May 2002, Bill Davidsen wrote:
>
> I think the reason which comes to mind is avoiding future problems. By
> having a single POSIX mode flag not only does the program not have to know
> about setting the "right" other bits today, but if we find that POSIX
> behaviour is needed in some other area in the future, the program doesn't
> need to be modified and recompiled, because the POSIX behaviour "is in
> there" for all things.

That's a nice argument in theory, but if you change the behaviour of
existing flags, you might fix some program for the real semantics, but you
might equally well _break_ some program that unwittingly depended on the
old semantics.

So I think your argument is fundamentally flawed. The binary has been
tested with the old behaviour, and assuming that you can "fix" existing
binaries by changing kernel behaviour is a seriously flawed argument.

Yes, it might work for some programs, but basically you're on very thin
ice.

Does Linux break stuff when absolutely required? Sure. But designing an
interface that _plans_ on changing semantics is just incredibly stupid,
and should absolutely not be done. Ever.

			Linus

Re: [RFC] POSIX personality

[jw schultz]

On Thu, May 23, 2002 at 10:09:28AM -0700, Linus Torvalds wrote:
> 
> 
> On Thu, 23 May 2002, Bill Davidsen wrote:
> >
> > I think the reason which comes to mind is avoiding future problems. By
> > having a single POSIX mode flag not only does the program not have to know
> > about setting the "right" other bits today, but if we find that POSIX
> > behaviour is needed in some other area in the future, the program doesn't
> > need to be modified and recompiled, because the POSIX behaviour "is in
> > there" for all things.
> 
> That's a nice argument in theory, but if you change the behaviour of
> existing flags, you might fix some program for the real semantics, but you
> might equally well _break_ some program that unwittingly depended on the
> old semantics.
> 
> So I think your argument is fundamentally flawed. The binary has been
> tested with the old behaviour, and assuming that you can "fix" existing
> binaries by changing kernel behaviour is a seriously flawed argument.
> 
> Yes, it might work for some programs, but basically you're on very thin
> ice.
> 
> Does Linux break stuff when absolutely required? Sure. But designing an
> interface that _plans_ on changing semantics is just incredibly stupid,
> and should absolutely not be done. Ever.
> 
> 			Linus

It seems to me that the biggest issue here is maintaining
POSIX behavior without having to modify application source
every time the flag set changes.

Perhaps a POSIX bitmask could be defined.

For a degree of binary compatibility a few unused flags
could be reserved and the POSIX bitmask include them
whether they had meaning yet or not. 

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: [RFC] POSIX personality

[Alan Cox]

> It seems to me that the biggest issue here is maintaining
> POSIX behavior without having to modify application source
> every time the flag set changes.

I don't think that is a big problem. Think about how it evolves over time


App calls pthread_foo   libpthreads/ngpt does all the work by emulation

	Add CLONE_somefoo

App calls pthread_foo	libpthreads/ngpt does all the work by emulation
			and doesnt set the flag

	New libpthreads

App calls pthread_foo	libpthreads/ngpt uses the kernel assists




The behaviour is good - it means that the new kernel/old library setup won't
break the emulation gunge by suddenely providing precise semantics itself

Re: [RFC] POSIX personality

[Bill Davidsen]

On Fri, 24 May 2002, jw schultz wrote:

> On Thu, May 23, 2002 at 10:09:28AM -0700, Linus Torvalds wrote:
> > 
> > 
> > On Thu, 23 May 2002, Bill Davidsen wrote:
> > >
> > > I think the reason which comes to mind is avoiding future problems. By
> > > having a single POSIX mode flag not only does the program not have to know
> > > about setting the "right" other bits today, but if we find that POSIX
> > > behaviour is needed in some other area in the future, the program doesn't
> > > need to be modified and recompiled, because the POSIX behaviour "is in
> > > there" for all things.
> > 
> > That's a nice argument in theory, but if you change the behaviour of
> > existing flags, you might fix some program for the real semantics, but you
> > might equally well _break_ some program that unwittingly depended on the
> > old semantics.
> > 
> > So I think your argument is fundamentally flawed. The binary has been
> > tested with the old behaviour, and assuming that you can "fix" existing
> > binaries by changing kernel behaviour is a seriously flawed argument.
> > 
> > Yes, it might work for some programs, but basically you're on very thin
> > ice.
> > 
> > Does Linux break stuff when absolutely required? Sure. But designing an
> > interface that _plans_ on changing semantics is just incredibly stupid,
> > and should absolutely not be done. Ever.
> > 
> > 			Linus
> 
> It seems to me that the biggest issue here is maintaining
> POSIX behavior without having to modify application source
> every time the flag set changes.
> 
> Perhaps a POSIX bitmask could be defined.
> 
> For a degree of binary compatibility a few unused flags
> could be reserved and the POSIX bitmask include them
> whether they had meaning yet or not. 

Obviously two ways of looking at things. Linus wants it to work the same
all the time, I want it to work the same on all POSIX systems, and if we
add (or find) a kernel change which isn't POSIX, and doing things in the
POSIX manner breaks the binary, the error is in the program. I have no
problem with bits to make little parts of Linux behave in a POSIX manner,
but I would like one POSIX bit which sets the most compliant behaviour
available.

I shouldn't have to recompile every time another instance of POSIX
behaviour is added, and I fail to see that bits are so precious that there
couldn't be one POSIX bit in addition to the bits allowing a monmgrel
hybred of POSIX and non-POSIX behaviour.

if (environ & OPTION_POSIX_FOO) { standard... } else { non-standard }
  is no faster than 
if (environ & (OPTION_POSIX | OPTION_POSIX_FOO)) ...
  and I fail to see why Linus is fighting the idea so hard. If correct
behaviour breaks my portable program it wasn't portable and the program is
in error. I set the bit, I have no complaint.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [RFC] POSIX personality

[george anzinger]

Dave McCracken wrote:
> 
> As part of improving support for POSIX multithreading I've been putting
> together some patches to allow more things to be shared between tasks.
> Right now this is accomplished via flags to clone() with one flag per
> resource to be shared.  This usually translates to a data structure pointed
> to out of task_struct, complete with reference count and lock.
> 
> In a discussion today an alternate idea was proposed by Ben LaHaise.  He
> suggested creating a POSIX personality, or execution domain.  This would
> take some pressure off the clone flag space as well as allowing some
> optimizations in the code. It could also be used in situations where
> POSIX-compatible behavior entails more than just sharing extra resources
> between tasks.
> 
> This would assume that the resources I'm sharing would only be useful for
> POSIX compatibility, but at this point it seems unlikely that anyone would
> want to share a subset of them.  The resources I'm currently working on
> include credentials, signals,  and timers, and there's a patch available
> for semaphore undo that could also be part of this mechanism.
> 
> Since you've made it this far my question to you all is this:  assuming
> that we do want improved POSIX compatibility does this sound like a
> reasonable way to add it?
> 
What you are proposing seem a bit vague.  I think that
CLONE_THREAD should group all the thread related stuff under
the one flag.  IMHO POSIX compatibility should not be off in
the corner as a step child, but rather should be the norm. 
The CLONE_THREAD flag would indicate to fork that it should
create a POSIX thread and set up the needed shared stuff.  I
rather image this to be a structure that each task_struct
points to, possibly with a usage count (but that is a
detail).  Each thread_struct would point to such a
structure, but processes that are not threaded would not be
sharing this area with other threads.

Is this close to what you had in mind?

-g
> 
> ======================================================================
> Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
> dmccr@us.ibm.com                                        T/L   678-3059
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
George Anzinger   george@mvista.com
High-res-timers: 
http://sourceforge.net/projects/high-res-timers/
Real time sched:  http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

Re: [RFC] POSIX personality

[Dave McCracken]

--On Tuesday, May 21, 2002 02:13:01 PM -0700 george anzinger
<george@mvista.com> wrote:

> What you are proposing seem a bit vague.  I think that
> CLONE_THREAD should group all the thread related stuff under
> the one flag.  IMHO POSIX compatibility should not be off in
> the corner as a step child, but rather should be the norm. 
> The CLONE_THREAD flag would indicate to fork that it should
> create a POSIX thread and set up the needed shared stuff.  I
> rather image this to be a structure that each task_struct
> points to, possibly with a usage count (but that is a
> detail).  Each thread_struct would point to such a
> structure, but processes that are not threaded would not be
> sharing this area with other threads.

That's a possibility I've considered, but I gather most of the kernel
community would prefer to see each resource have its own flag to clone() so
applications can pick and choose which ones to share.  I'm guessing that in
most cases multithreaded apps will choose all flags.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [RFC] POSIX personality

[Andi Kleen]

Dave McCracken <dmccr@us.ibm.com> writes:

> --On Tuesday, May 21, 2002 01:52:37 PM -0700 Linus Torvalds
> <torvalds@transmeta.com> wrote:
> 
> > I don't see any reason to start using some fixed-mode semantics without 
> > seeing some stronger arguments on exactly why that would be a good idea. 
> > We have used up 11 of 24 bits (and more can be made available) over the 
> > last five years, and there are no obvious inefficiencies that I can see.
> 
> Ok, sounds reasonable.  I'll add the bits as I go, then.

One reason for it would be that it would be more efficient. All the various
shared state needed for POSIX thread group emulation could be put into a 
single structure with a single reference count.

With clone flags you need one pointer in task_struct per flag and
handling of the reference count for each data structure and allocation/freeing
from various slabs for a real fork.
(basically lots of atomic operations at fork time + bloating of task_struct) 

-Andi

Re: [RFC] POSIX personality

[Linus Torvalds]

On 21 May 2002, Andi Kleen wrote:
> 
> One reason for it would be that it would be more efficient. All the various
> shared state needed for POSIX thread group emulation could be put into a 
> single structure with a single reference count.

Now, that's a separate issue - the issue of the exact _granularity_ of the 
bits, and how you group things together.

On that front, I don't have any strong feelings - but I suspect that it 
almost always ends up being fairly obvious when it is "right" to group 
things together, and when it isn't.

For example, we probably could have had just one bit for (FS | FILES), and 
the same is probably true of (SIGHAND | THREAD), but on the whole we 
haven't really had any gray areas when it comes to the grouping. And I 
don't see any coming up.

Does that mean that we might have a CLONE_POSIXDAMAGE that just covers all
the strange POSIX stuff that make no sense anywhere else? Maybe. But I'd
want that to be just another bit with the same semantic behaviour as the
existing ones, _not_ be some external "POSIX personality".

		Linus

Re: [RFC] POSIX personality

[Dave McCracken]

--On Tuesday, May 21, 2002 04:08:52 PM -0700 Linus Torvalds
<torvalds@transmeta.com> wrote:

>> 
>> One reason for it would be that it would be more efficient. All the
>> various shared state needed for POSIX thread group emulation could be
>> put into a  single structure with a single reference count.
> 
> Now, that's a separate issue - the issue of the exact _granularity_ of
> the  bits, and how you group things together.
> 
> On that front, I don't have any strong feelings - but I suspect that it 
> almost always ends up being fairly obvious when it is "right" to group 
> things together, and when it isn't.
> 
> For example, we probably could have had just one bit for (FS | FILES),
> and  the same is probably true of (SIGHAND | THREAD), but on the whole we 
> haven't really had any gray areas when it comes to the grouping. And I 
> don't see any coming up.
> 
> Does that mean that we might have a CLONE_POSIXDAMAGE that just covers all
> the strange POSIX stuff that make no sense anywhere else? Maybe. But I'd
> want that to be just another bit with the same semantic behaviour as the
> existing ones, _not_ be some external "POSIX personality".

I've been thinking along those lines myself.  At this point I'd suggest we
implement them as separate, then if in the future no one ever uses them
separately we can consider combining them.  I know this can raise some
backward compatibility issues but in theory if anyone cares we wouldn't do
it.

Dave McCracken

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [RFC] POSIX personality

[Michael Sinz]

Dave McCracken wrote:
> --On Tuesday, May 21, 2002 04:08:52 PM -0700 Linus Torvalds
> <torvalds@transmeta.com> wrote:
> 
> 
>>>One reason for it would be that it would be more efficient. All the
>>>various shared state needed for POSIX thread group emulation could be
>>>put into a  single structure with a single reference count.
>>
>>Now, that's a separate issue - the issue of the exact _granularity_ of
>>the  bits, and how you group things together.
>>
>>On that front, I don't have any strong feelings - but I suspect that it 
>>almost always ends up being fairly obvious when it is "right" to group 
>>things together, and when it isn't.
>>
>>For example, we probably could have had just one bit for (FS | FILES),
>>and  the same is probably true of (SIGHAND | THREAD), but on the whole we 
>>haven't really had any gray areas when it comes to the grouping. And I 
>>don't see any coming up.
>>
>>Does that mean that we might have a CLONE_POSIXDAMAGE that just covers all
>>the strange POSIX stuff that make no sense anywhere else? Maybe. But I'd
>>want that to be just another bit with the same semantic behaviour as the
>>existing ones, _not_ be some external "POSIX personality".
> 
> 
> I've been thinking along those lines myself.  At this point I'd suggest we
> implement them as separate, then if in the future no one ever uses them
> separately we can consider combining them.  I know this can raise some
> backward compatibility issues but in theory if anyone cares we wouldn't do
> it.

I would be worried about the future compatibility here.  It would be easier
to be compatible to start with a single bit and then add individual bits for
those features that need to be broken out when it is know to be needed.
Folding the bits back in is not as easy as you now have to still support
the individual but yet unneeded.

-- 
Michael Sinz ---- Worldgate Communications ---- msinz@wgate.com
A master's secrets are only as good as
	the master's ability to explain them to others.

Re: [RFC] POSIX personality

[Dave McCracken]

--On Tuesday, May 28, 2002 01:26:28 PM -0400 Michael Sinz <msinz@wgate.com>
wrote:

>> I've been thinking along those lines myself.  At this point I'd suggest
>> we implement them as separate, then if in the future no one ever uses
>> them separately we can consider combining them.  I know this can raise
>> some backward compatibility issues but in theory if anyone cares we
>> wouldn't do it.
> 
> I would be worried about the future compatibility here.  It would be
> easier to be compatible to start with a single bit and then add
> individual bits for those features that need to be broken out when it is
> know to be needed. Folding the bits back in is not as easy as you now
> have to still support the individual but yet unneeded.

That's a good point.  But at this point I don't see any items where we can
say up front that everyone will want all or none of them.  I suspect we'll
just have to live with them as separate flags.

Dave

======================================================================
Dave McCracken          IBM Linux Base Kernel Team      1-512-838-3059
dmccr@us.ibm.com                                        T/L   678-3059

Re: [RFC] POSIX personality

[Peter Chubb]

>>>>> "Andi" == Andi Kleen <ak@suse.de> writes:

Andi> Dave McCracken <dmccr@us.ibm.com> writes:
>> --On Tuesday, May 21, 2002 01:52:37 PM -0700 Linus Torvalds
>> <torvalds@transmeta.com> wrote:
>> 
>> > I don't see any reason to start using some fixed-mode semantics
>> without > seeing some stronger arguments on exactly why that would
>> be a good idea. > We have used up 11 of 24 bits (and more can be
>> made available) over the > last five years, and there are no
>> obvious inefficiencies that I can see.
>> 
>> Ok, sounds reasonable.  I'll add the bits as I go, then.

Andi> One reason for it would be that it would be more efficient. All
Andi> the various shared state needed for POSIX thread group emulation
Andi> could be put into a single structure with a single reference
Andi> count.

Andi> With clone flags you need one pointer in task_struct per flag
Andi> and handling of the reference count for each data structure and
Andi> allocation/freeing from various slabs for a real fork.
Andi> (basically lots of atomic operations at fork time + bloating of
Andi> task_struct)


With one minor restriction, you could do it the same way SGI's IRIX
handled sproc() --- have a   struct shared   that was pointed to by the
task structure, that was then partially or completely populated
depending on the sharing flags.  The   struct shared   can also have a
list of tasks that share stuff in it.  Each task has a set of flags to
say what's being shared.

You'd have to disallow sharing things that a task's parent doesn't
share with the task, if the parent and the child are part of the same
share group. 

Peter C