Linux and Posix compliance

Linux and Posix compliance

[Ramsey Wally Contr AFRL/IFEB]

To all:

This may not be the appropriate list for this, but can anyone point me to
references on POSIX compliance and Linux? I am just getting started on this.

thanks,

Wally Ramsey
ramseyw@rl.af.mil

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[James A Griffin]

Ramsey Wally Contr AFRL/IFEB wrote:
> 
> To all:
> 
> This may not be the appropriate list for this, but can anyone point me to
> references on POSIX compliance and Linux? I am just getting started on this.
> 
[snip]

One thing that I have noticed for years is the boot time message "POSIX
conformance testing by UNIFIX".  The message appears just after the CPU
testing and before the PCI: Probe.  UNIFIX is a UK company, IIRC.  What
the results of the "conformance testing" are, I do not know.

Does anyone on the list know?

Regards,
Jim

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[Ulrich Drepper]

James A Griffin <agriffin@cpcug.org> writes:

> One thing that I have noticed for years is the boot time message "POSIX
> conformance testing by UNIFIX".  The message appears just after the CPU
> testing and before the PCI: Probe.  UNIFIX is a UK company, IIRC.  What
> the results of the "conformance testing" are, I do not know.

I think (and hope) they've finally removed this output.  It never was
true and is anyway completely outdated.

There hasn't been any formal POSIX testing done.  The reason is
simple: who'd pay this?  POSIX testing is done by the certification
labs which charge enormous amounts of money for this.

The kernel people has done tests based on available standard tests
occasionally.  I've done for the C library quite some testing and
wrote many tests myself.  This still does not guarantee compliance,
though.


As for use in government and military.  The current route seems to be
to get for Linux the same kind of exception which exists for NT.  NT
does not conform to POSIX either and can be used.  You might want to
check your appropriate standardization bureau about the status of
this.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

RE: Linux and Posix compliance

[Weaver, Mike D (N-CSC)]

Ulrich,

You are wrong on a few minor points.

1) NT V3.51 passed successfully the test for POSIX 1003.1 (at the time that
was all that was complete)  they never made any effort to accomplish
anything else.  MKS went to some effort to provide 1003.2 when it became
complete and did actually provide a full 1003.1/1003.2 set of functionality
and when combined with the set of tools they provided was a superset of the
POSIX functionality. As for compliance beyond 3.51 you may be correct, (ie
4.0, 2000, etc) who knows... and I'm sure the testing will only continue if
absolutely necessary)  Bottom line is that it will only continue as long as
the requirement (read: sales volume) drives the continued testing.

2) The tests do not cost a fortune since the IEEE still (I believe) owns
them, what costs a fortune is the certification by an independent body that
a candidate platform meets the tests.  The bar was always set high to
preclude the unscrupulous vendor from cheating, as many would be prone to
do.  If one wanted to test Linux it would be fairly cheap if not free to do
the testing.  And as I mailed Wally privately it would likely come pretty
close to passing (my subjective guess would be that it would be about 80%
compliant) with no changes, and with some work around certain IPC constructs
could probably be made to comply.

3) As you say NT has become an exception throughout the US government.  You
are so correct.  This is not however by ANY written or spoken exception
process legal or otherwise.  They simply are selectively ignoring their own
mandates (another point I made privately with Wally) for their own reasons
of narrowing the bidding field.  The process has been totally and completely
driven by the sales volume of the MS platforms, and MS's grudging acceptance
of government requirements for things like security, posix compliance, etc.

As for the tests, compliance of platforms being meaningless. You nailed it!
The appropriate body in this case is NIST, which is in bed with the IEEE,
however NIST, and in Wally's case the USAF can ignore standards compliance
(or in NT's case the lack thereof) on a completely arbitrary basis to meet
the 'needs' of the organization (or to rule out a candidate
product/solution).

Cheers,

mdw ;-)

PS.  I also sent Wally a couple of pointers on the
http://www.freestandards.org/ldps/
efforts of the Linux standardization community.

-----Original Message-----
From: Ulrich Drepper [mailto:drepper@redhat.com]
Sent: Monday, July 16, 2001 2:11 PM
To: James A Griffin
Cc: Ramsey Wally Contr AFRL/IFEB; 'NSA SELinux Mailing List'
Subject: Re: Linux and Posix compliance


James A Griffin <agriffin@cpcug.org> writes:

> One thing that I have noticed for years is the boot time message "POSIX
> conformance testing by UNIFIX".  The message appears just after the CPU
> testing and before the PCI: Probe.  UNIFIX is a UK company, IIRC.  What
> the results of the "conformance testing" are, I do not know.

I think (and hope) they've finally removed this output.  It never was
true and is anyway completely outdated.

There hasn't been any formal POSIX testing done.  The reason is
simple: who'd pay this?  POSIX testing is done by the certification
labs which charge enormous amounts of money for this.

The kernel people has done tests based on available standard tests
occasionally.  I've done for the C library quite some testing and
wrote many tests myself.  This still does not guarantee compliance,
though.


As for use in government and military.  The current route seems to be
to get for Linux the same kind of exception which exists for NT.  NT
does not conform to POSIX either and can be used.  You might want to
check your appropriate standardization bureau about the status of
this.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

--
You have received this message because you are subscribed to the selinux
list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov
with
the words "unsubscribe selinux" without quotes as the message.

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[Ulrich Drepper]

"Weaver, Mike D (N-CSC)" <mike.d.weaver@lmco.com> writes:

> 1) NT V3.51 passed successfully the test for POSIX 1003.1 (at the time that
> was all that was complete)

That's the point.  Who cares about the old .1/.2?  All applications
are using features from the additions.  There is no such support and
this is why there is the exception for NT.

> 2) The tests do not cost a fortune since the IEEE still (I believe) owns
> them,

There exists no test suite for the current standard, leave alone a
free one.  Standard labs are working on updates but there isn't
certification offered yet.  All which is freely available is a test
for a subset for the old .1 spec.  This is completely inadequate and
the test suite itself is so full of bugs that it's not even funny
(bugs = assumptions made which are not part of the standard).  The
IEEE test suite is AFAIK directly and automatically derived from the
2003 standards which were not updated since the first release.

I've tried getting test suites and all you hear from the OpenGroup or
similar organizations is how much they want to charge you.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[James A Griffin]

Ulrich Drepper wrote:
> 
> James A Griffin <agriffin@cpcug.org> writes:
> 
> > One thing that I have noticed for years is the boot time message "POSIX
> > conformance testing by UNIFIX".  The message appears just after the CPU
> > testing and before the PCI: Probe.  UNIFIX is a UK company, IIRC.  What
> > the results of the "conformance testing" are, I do not know.
> 
> I think (and hope) they've finally removed this output.  It never was
> true and is anyway completely outdated.
> 
The "POSIX conformance testing by UNIFIX" message is still in the
kernel, see:

Linux version 2.4.7-pre5 (root@sparta.athena.inc) (gcc version 2.96
20000731 (Red Hat Linux 7.1 2.96-85)) #29 Tue Jul 10 21:08:23 EDT 2001
[snip]
CPU: AMD-K5(tm) Processor stepping 04
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
PCI: PCI BIOS revision 2.10 entry at 0xf04e0, last bus=0
PCI: Using configuration type 1
[snip]

May be time to send a note to Allen or Linus.

[snip discussion of formal testing and issues]

Regards,
Jim

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[Christoph Hellwig]

On Mon, Jul 16, 2001 at 01:10:43PM -0700, Ulrich Drepper wrote:
> James A Griffin <agriffin@cpcug.org> writes:
> 
> > One thing that I have noticed for years is the boot time message "POSIX
> > conformance testing by UNIFIX".  The message appears just after the CPU
> > testing and before the PCI: Probe.  UNIFIX is a UK company, IIRC.  What
> > the results of the "conformance testing" are, I do not know.
> 
> I think (and hope) they've finally removed this output.  It never was
> true and is anyway completely outdated.

It's still there.  And though it is outdated I bet it was once right.

> There hasn't been any formal POSIX testing done.  The reason is
> simple: who'd pay this?

UNIFIX, as the message shows.  UNIFIX is (or was?) a German Linux
distributor, located in Braunschwieg (100km from my home, so I could
take a look if no one believes in this ;)).

In 1995 or 1996 they sold a UNIFIX 2.0 distribution which was heavily
modified not only to pass the posix.1 and posix.2 (IIRC) conformance
test, but also too look more sysvish (I remeber yhey modified e.g.
ps to take ps -elf for long output instead of ps aux).

	Christoph

-- 
Whip me.  Beat me.  Make me maintain AIX.

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[Andrew Josey]

Ulrich writes in response to another posting:

> > 2) The tests do not cost a fortune since the IEEE still (I believe) owns
> > them,
>
>There exists no test suite for the current standard, leave alone a
>free one.  Standard labs are working on updates but there isn't
>certification offered yet.  All which is freely available is a test
>for a subset for the old .1 spec.  This is completely inadequate and
>the test suite itself is so full of bugs that it's not even funny
>(bugs = assumptions made which are not part of the standard).  The
>IEEE test suite is AFAIK directly and automatically derived from the
>2003 standards which were not updated since the first release.

The current standard is IEEE Std 1003.1-1990 plus its
amendments, the nature of the amendments process is that
it is additive and so what is out there ought to be current.
In the case of the NIST PCTS test suite, this was lasted updated in 1995,
and is known to be problematic.   This is not the only
test suite for POSIX, and The Open Group has test suites
for all of POSIX, and makes freely available a core set.
The equivalent test suite  to the NIST PCTS for ISO/IEC 9945-1:1990
is known as the VSX-PCTS test suite and this is regularly maintained
and updated.


>

>I've tried getting test suites and all you hear from the OpenGroup or
>similar organizations is how much they want to charge you.
>
I think this is a little unfair -- The Open Group makes freely available many
test suites as part of its support for the community. See
http://www.opengroup.org/testing/downloads.html,
which has tests for IEEE POSIX.1-1990 (VSX-PCTS), POSIX.2-1992 (VSC-lite) and
POSIX.1c-1995 (VSTH-lite) (also the LSB FHS tests, the LSB-VSX,
X Window System, Motif, Corba and various test frameworks and tools).

The VSX-PCTS is being used as the core part of the LSB test
and certification activities, where The Open Group is contributing
new testcases beyond the POSIX subset in the LSB-OS testsuite.

Yes, we do charge for some of our other test suites for other areas
such as UNIX certification which have greater test coverage --  we operate
as a  not-for-profit and selling licenses and support services is how
we pay our salaries.

We are commencing work on the update to the tests for the forthcoming
revision to POSIX (see http://www.opengroup.org/austin/ for the
spec) and do plan to continue to make a  POSIX subset available.
If any folks are interested in assisting with the funding for
this effort let me know.

best regards
Andrew


-----
Andrew Josey                                The Open Group
Director, Server Platforms                  Apex Plaza,Forbury Road,
Email: a.josey@opengroup.org                Reading,Berks.RG1 1AX,England

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Linux and Posix compliance

[Andrew Josey]

>2) The tests do not cost a fortune since the IEEE still (I believe) owns
>them, what costs a fortune is the certification by an independent body that
>a candidate platform meets the tests.  The bar was always set high to
>preclude the unscrupulous vendor from cheating, as many would be prone to
>do.  If one wanted to test Linux it would be fairly cheap if not free to do
>the testing.  And as I mailed Wally privately it would likely come pretty
>close to passing (my subjective guess would be that it would be about 80%
>compliant) with no changes, and with some work around certain IPC constructs
>could probably be made to comply.

I believe the fee structure setup with the NVLAP accredited labs
(for NIST) typically included some aspect of consultancy to complete the
paperwork of the conformance documentation. The actual certification
fees were typically much lower (the order of a few hundred dollars).

Today you can pick up the VSX-PCTS, pre-configured for Linux
in the form of the LSB-VSX test suite being used as part of
the Linux Standard Base testing efforts, and run this quite easily
on any machine.  Unofficial results are available from
the LSB test website at http://www.linuxbase.org/
The Open Group would be happy to certify an application for
compliance using the VSX-PCTS and charge just an administrative
processing fee

best regards
Andrew


-----
Andrew Josey                                The Open Group
Director, Server Platforms                  Apex Plaza,Forbury Road,
Email: a.josey@opengroup.org                Reading,Berks.RG1 1AX,England
Tel:   +44 118 9508311 ext 2250             Fax: +44 118 9500110
Mobile: +44 774 015 5794

--
You have received this message because you are subscribed to the selinux list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.