new architectures, time_t __kernel_long_t

new architectures, time_t __kernel_long_t

[Arnd Bergmann]

Since we are in the review phase for two new architectures that we
should be merging into the mainline kernel, I think we need to
come up with a solution for making them use a proper 64-bit time_t.

Right now, the only 32-bit user space interface we have to use 64-bit
time_t is the x32 side of x86-64, and that works by overriding all
"long" sized types to be 64 bit at the ABI level, which we don't
want for pure 32 bit architectures.
 
I would suggest we make 64-bit time_t, clock_t and suseconds_t the
default in include/asm-generic/posix_types.h and change all existing
32 bit users of that file to provide an override to define these
as "long".

The other types that are used as 64 bit on x32 are ino_t, nlink_t,
size_t, ssize_t, ptrdiff_t, and off_t.

Obviously, we want to use 64-bit off_t, but this is achieved already
through loff_t, which is used in all places in the asm-generic
ABI anyway (the syscalls using off_t are stripped out). I don't
think we want to have the other ones set to 64 bit on ARC or Meta,
although I'm not 100% sure about ino_t and nlink_t.

	Arnd

Re: new architectures, time_t __kernel_long_t

[Geert Uytterhoeven]

On Wed, Nov 14, 2012 at 1:18 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> size_t, ssize_t, ptrdiff_t, and off_t.
>
> Obviously, we want to use 64-bit off_t, but this is achieved already
> through loff_t, which is used in all places in the asm-generic
> ABI anyway (the syscalls using off_t are stripped out). I don't

Indeed.

> think we want to have the other ones set to 64 bit on ARC or Meta,
> although I'm not 100% sure about ino_t and nlink_t.

Is ino_t related to inode numbers stored in on-disk file system metadata?

Currently, nlink_t is unconditionally __u32, even on 64-bit.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
> Since we are in the review phase for two new architectures that we
> should be merging into the mainline kernel, I think we need to
> come up with a solution for making them use a proper 64-bit time_t.
>
> Right now, the only 32-bit user space interface we have to use 64-bit
> time_t is the x32 side of x86-64, and that works by overriding all
> "long" sized types to be 64 bit at the ABI level, which we don't
> want for pure 32 bit architectures.

Sort of.  Either way, the kernel headers aren't really x32-clean yet, so 
we have an opportunity to do things more cleanly as we are implementing 
this.

> I would suggest we make 64-bit time_t, clock_t and suseconds_t the
> default in include/asm-generic/posix_types.h and change all existing
> 32 bit users of that file to provide an override to define these
> as "long".

We also need snseconds_t: POSIX stupidly enough demands that the 
nanosecond field in struct timespec is "long", but that is really not 
appropriate.  Unfortunately there are existing architectures (e.g. 
SPARC) where we can't use suseconds_t either, so we are trying to push a 
new snseconds_t type into POSIX.

> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> size_t, ssize_t, ptrdiff_t, and off_t.

ptrdiff_t obviously not... although __kernel_ptrdiff_t is (as the 
pointers are 64 bits in kernel space.)  Similarly, size_t and ssize_t 
are only 32 bits in user space and we override them for compatibility 
with the native kernel.

> Obviously, we want to use 64-bit off_t, but this is achieved already
> through loff_t, which is used in all places in the asm-generic
> ABI anyway (the syscalls using off_t are stripped out). I don't
> think we want to have the other ones set to 64 bit on ARC or Meta,
> although I'm not 100% sure about ino_t and nlink_t.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
> Since we are in the review phase for two new architectures that we
> should be merging into the mainline kernel, I think we need to
> come up with a solution for making them use a proper 64-bit time_t.
>
> Right now, the only 32-bit user space interface we have to use 64-bit
> time_t is the x32 side of x86-64, and that works by overriding all
> "long" sized types to be 64 bit at the ABI level, which we don't
> want for pure 32 bit architectures.

Sort of.  Either way, the kernel headers aren't really x32-clean yet, so 
we have an opportunity to do things more cleanly as we are implementing 
this.

> I would suggest we make 64-bit time_t, clock_t and suseconds_t the
> default in include/asm-generic/posix_types.h and change all existing
> 32 bit users of that file to provide an override to define these
> as "long".

We also need snseconds_t: POSIX stupidly enough demands that the 
nanosecond field in struct timespec is "long", but that is really not 
appropriate.  Unfortunately there are existing architectures (e.g. 
SPARC) where we can't use suseconds_t either, so we are trying to push a 
new snseconds_t type into POSIX.

> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> size_t, ssize_t, ptrdiff_t, and off_t.

ptrdiff_t obviously not... although __kernel_ptrdiff_t is (as the 
pointers are 64 bits in kernel space.)  Similarly, size_t and ssize_t 
are only 32 bits in user space and we override them for compatibility 
with the native kernel.

> Obviously, we want to use 64-bit off_t, but this is achieved already
> through loff_t, which is used in all places in the asm-generic
> ABI anyway (the syscalls using off_t are stripped out). I don't
> think we want to have the other ones set to 64 bit on ARC or Meta,
> although I'm not 100% sure about ino_t and nlink_t.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: new architectures, time_t __kernel_long_t

[Arnd Bergmann]

On Wednesday 14 November 2012, H. Peter Anvin wrote:
> On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
> > Since we are in the review phase for two new architectures that we
> > should be merging into the mainline kernel, I think we need to
> > come up with a solution for making them use a proper 64-bit time_t.
> >
> > Right now, the only 32-bit user space interface we have to use 64-bit
> > time_t is the x32 side of x86-64, and that works by overriding all
> > "long" sized types to be 64 bit at the ABI level, which we don't
> > want for pure 32 bit architectures.
> 
> Sort of.  Either way, the kernel headers aren't really x32-clean yet, so 
> we have an opportunity to do things more cleanly as we are implementing 
> this.

Ah, I didn't know that. How does one build an x32 libc then?

> > I would suggest we make 64-bit time_t, clock_t and suseconds_t the
> > default in include/asm-generic/posix_types.h and change all existing
> > 32 bit users of that file to provide an override to define these
> > as "long".
> 
> We also need snseconds_t: POSIX stupidly enough demands that the 
> nanosecond field in struct timespec is "long", but that is really not 
> appropriate.  Unfortunately there are existing architectures (e.g. 
> SPARC) where we can't use suseconds_t either, so we are trying to push a 
> new snseconds_t type into POSIX.

Right.

> > The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> > size_t, ssize_t, ptrdiff_t, and off_t.
> 
> ptrdiff_t obviously not... although __kernel_ptrdiff_t is (as the 
> pointers are 64 bits in kernel space.)  Similarly, size_t and ssize_t 
> are only 32 bits in user space and we override them for compatibility 
> with the native kernel.

For ARC and Meta, the pointers in kernel space are never 64 bit. For x32,
they could in theory be 32 bit if one was to do a native x32 kernel port
(which I hope nobody will do).

The __kernel_* types are never about what the kernel actually has (user space
should not care), they are about what the ABI defines that are are using.

We could in theory define all the above (and __kernel_ulong_t) to be 64 bit
wide on all new architectures, which would allow doing 64 bit port later
without a compat layer, but that benefit would be far outweighed by the
complexity and cost of passing 64-bit arguments in two registers for a lot
of syscalls.

	Arnd

Re: new architectures, time_t __kernel_long_t

[Arnd Bergmann]

On Wednesday 14 November 2012, H. Peter Anvin wrote:
> On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
> > Since we are in the review phase for two new architectures that we
> > should be merging into the mainline kernel, I think we need to
> > come up with a solution for making them use a proper 64-bit time_t.
> >
> > Right now, the only 32-bit user space interface we have to use 64-bit
> > time_t is the x32 side of x86-64, and that works by overriding all
> > "long" sized types to be 64 bit at the ABI level, which we don't
> > want for pure 32 bit architectures.
> 
> Sort of.  Either way, the kernel headers aren't really x32-clean yet, so 
> we have an opportunity to do things more cleanly as we are implementing 
> this.

Ah, I didn't know that. How does one build an x32 libc then?

> > I would suggest we make 64-bit time_t, clock_t and suseconds_t the
> > default in include/asm-generic/posix_types.h and change all existing
> > 32 bit users of that file to provide an override to define these
> > as "long".
> 
> We also need snseconds_t: POSIX stupidly enough demands that the 
> nanosecond field in struct timespec is "long", but that is really not 
> appropriate.  Unfortunately there are existing architectures (e.g. 
> SPARC) where we can't use suseconds_t either, so we are trying to push a 
> new snseconds_t type into POSIX.

Right.

> > The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> > size_t, ssize_t, ptrdiff_t, and off_t.
> 
> ptrdiff_t obviously not... although __kernel_ptrdiff_t is (as the 
> pointers are 64 bits in kernel space.)  Similarly, size_t and ssize_t 
> are only 32 bits in user space and we override them for compatibility 
> with the native kernel.

For ARC and Meta, the pointers in kernel space are never 64 bit. For x32,
they could in theory be 32 bit if one was to do a native x32 kernel port
(which I hope nobody will do).

The __kernel_* types are never about what the kernel actually has (user space
should not care), they are about what the ABI defines that are are using.

We could in theory define all the above (and __kernel_ulong_t) to be 64 bit
wide on all new architectures, which would allow doing 64 bit port later
without a compat layer, but that benefit would be far outweighed by the
complexity and cost of passing 64-bit arguments in two registers for a lot
of syscalls.

	Arnd

Re: new architectures, time_t __kernel_long_t

[H.J. Lu]

On Thu, Nov 15, 2012 at 1:14 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Wednesday 14 November 2012, H. Peter Anvin wrote:
>> On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
>> > Since we are in the review phase for two new architectures that we
>> > should be merging into the mainline kernel, I think we need to
>> > come up with a solution for making them use a proper 64-bit time_t.
>> >
>> > Right now, the only 32-bit user space interface we have to use 64-bit
>> > time_t is the x32 side of x86-64, and that works by overriding all
>> > "long" sized types to be 64 bit at the ABI level, which we don't
>> > want for pure 32 bit architectures.
>>
>> Sort of.  Either way, the kernel headers aren't really x32-clean yet, so
>> we have an opportunity to do things more cleanly as we are implementing
>> this.
>
> Ah, I didn't know that. How does one build an x32 libc then?

Glibc has been providing its own types for years.
Kernel provided types used to be wrong for ia32
on x86-64.

-- 
H.J.

Re: new architectures, time_t __kernel_long_t

[Arnd Bergmann]

On Thursday 15 November 2012, H.J. Lu wrote:
> On Thu, Nov 15, 2012 at 1:14 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Wednesday 14 November 2012, H. Peter Anvin wrote:
> >> On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
> >> > Since we are in the review phase for two new architectures that we
> >> > should be merging into the mainline kernel, I think we need to
> >> > come up with a solution for making them use a proper 64-bit time_t.
> >> >
> >> > Right now, the only 32-bit user space interface we have to use 64-bit
> >> > time_t is the x32 side of x86-64, and that works by overriding all
> >> > "long" sized types to be 64 bit at the ABI level, which we don't
> >> > want for pure 32 bit architectures.
> >>
> >> Sort of.  Either way, the kernel headers aren't really x32-clean yet, so
> >> we have an opportunity to do things more cleanly as we are implementing
> >> this.
> >
> > Ah, I didn't know that. How does one build an x32 libc then?
> 
> Glibc has been providing its own types for years.
> Kernel provided types used to be wrong for ia32
> on x86-64.

What about ioctls and other calls then that actually do rely on the
kernel headers and use the __kernel_*_t types?

	Arnd

Re: new architectures, time_t __kernel_long_t

[H.J. Lu]

On Thu, Nov 15, 2012 at 6:36 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Thursday 15 November 2012, H.J. Lu wrote:
>> On Thu, Nov 15, 2012 at 1:14 AM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Wednesday 14 November 2012, H. Peter Anvin wrote:
>> >> On 11/14/2012 04:18 AM, Arnd Bergmann wrote:
>> >> > Since we are in the review phase for two new architectures that we
>> >> > should be merging into the mainline kernel, I think we need to
>> >> > come up with a solution for making them use a proper 64-bit time_t.
>> >> >
>> >> > Right now, the only 32-bit user space interface we have to use 64-bit
>> >> > time_t is the x32 side of x86-64, and that works by overriding all
>> >> > "long" sized types to be 64 bit at the ABI level, which we don't
>> >> > want for pure 32 bit architectures.
>> >>
>> >> Sort of.  Either way, the kernel headers aren't really x32-clean yet, so
>> >> we have an opportunity to do things more cleanly as we are implementing
>> >> this.
>> >
>> > Ah, I didn't know that. How does one build an x32 libc then?
>>
>> Glibc has been providing its own types for years.
>> Kernel provided types used to be wrong for ia32
>> on x86-64.
>
> What about ioctls and other calls then that actually do rely on the
> kernel headers and use the __kernel_*_t types?
>

Glibc defines __syscall_slong_t and __syscall_ulong_t.

-- 
H.J.

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

On 11/15/2012 06:36 AM, Arnd Bergmann wrote:
>>
>> Glibc has been providing its own types for years.
>> Kernel provided types used to be wrong for ia32
>> on x86-64.
>
> What about ioctls and other calls then that actually do rely on the
> kernel headers and use the __kernel_*_t types?
>

Now, glibc *shouldn't* have to do that... it is an indication of failure 
on our part.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: new architectures, time_t __kernel_long_t

[Al Viro]

On Wed, Nov 14, 2012 at 12:18:01PM +0000, Arnd Bergmann wrote:

> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> size_t, ssize_t, ptrdiff_t, and off_t.

*Kernel-side* we should not give a damn about the userland nlink_t, period.
Making it architecture-dependent had been a bad mistake that essentially
made nlink_t useless for the kernel.  That mistake had been fixed; please,
do not bring it back.  If some userland structure needs to include a field
encoding nlink_t values, please use an explicitly-sized type when refering
to it kernel-side.

The same should've been true for mode_t, but for historical reasons we
are using umode_t for just about everything and IMO we should kill the
last references to mode_t anywhere kernel-side (again, explicitly-sized
types for userland st_mode and friends on the last few architectures
still refering to mode_t there) and just rename umode_t to mode_t; I'm
sick and tired of playing whack-a-mole with code using (arch-dependent)
mode_t for kernel data.  And no, it's not always harmless - we had rather
ugly bugs based on that.

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

On 12/20/2012 08:57 PM, Al Viro wrote:
> On Wed, Nov 14, 2012 at 12:18:01PM +0000, Arnd Bergmann wrote:
> 
>> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
>> size_t, ssize_t, ptrdiff_t, and off_t.
> 
> *Kernel-side* we should not give a damn about the userland nlink_t, period.
> Making it architecture-dependent had been a bad mistake that essentially
> made nlink_t useless for the kernel.  That mistake had been fixed; please,
> do not bring it back.  If some userland structure needs to include a field
> encoding nlink_t values, please use an explicitly-sized type when refering
> to it kernel-side.
> 

We should never use userland types per se.  We can use __kernel_*_t
typedefs to make the kernel headers neater if it makes sense, but that
is often not even necessary.

> The same should've been true for mode_t, but for historical reasons we
> are using umode_t for just about everything and IMO we should kill the
> last references to mode_t anywhere kernel-side (again, explicitly-sized
> types for userland st_mode and friends on the last few architectures
> still refering to mode_t there) and just rename umode_t to mode_t; I'm
> sick and tired of playing whack-a-mole with code using (arch-dependent)
> mode_t for kernel data.  And no, it's not always harmless - we had rather
> ugly bugs based on that.
> 

Re: new architectures, time_t __kernel_long_t

[Al Viro]

On Thu, Dec 20, 2012 at 09:00:27PM -0800, H. Peter Anvin wrote:
> On 12/20/2012 08:57 PM, Al Viro wrote:
> > On Wed, Nov 14, 2012 at 12:18:01PM +0000, Arnd Bergmann wrote:
> > 
> >> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
> >> size_t, ssize_t, ptrdiff_t, and off_t.
> > 
> > *Kernel-side* we should not give a damn about the userland nlink_t, period.
> > Making it architecture-dependent had been a bad mistake that essentially
> > made nlink_t useless for the kernel.  That mistake had been fixed; please,
> > do not bring it back.  If some userland structure needs to include a field
> > encoding nlink_t values, please use an explicitly-sized type when refering
> > to it kernel-side.
> > 
> 
> We should never use userland types per se.  We can use __kernel_*_t
> typedefs to make the kernel headers neater if it makes sense, but that
> is often not even necessary.

... as long as we do not have typedef __kernel_foo_t foo_t in linux/types.h.

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

On 12/20/2012 09:02 PM, Al Viro wrote:
> On Thu, Dec 20, 2012 at 09:00:27PM -0800, H. Peter Anvin wrote:
>> On 12/20/2012 08:57 PM, Al Viro wrote:
>>> On Wed, Nov 14, 2012 at 12:18:01PM +0000, Arnd Bergmann wrote:
>>>
>>>> The other types that are used as 64 bit on x32 are ino_t, nlink_t,
>>>> size_t, ssize_t, ptrdiff_t, and off_t.
>>>
>>> *Kernel-side* we should not give a damn about the userland nlink_t, period.
>>> Making it architecture-dependent had been a bad mistake that essentially
>>> made nlink_t useless for the kernel.  That mistake had been fixed; please,
>>> do not bring it back.  If some userland structure needs to include a field
>>> encoding nlink_t values, please use an explicitly-sized type when refering
>>> to it kernel-side.
>>>
>>
>> We should never use userland types per se.  We can use __kernel_*_t
>> typedefs to make the kernel headers neater if it makes sense, but that
>> is often not even necessary.
>
> ... as long as we do not have typedef __kernel_foo_t foo_t in linux/types.h.
>

In the case of things like nlink_t and dev_t I would suggest we 
explicitly call out the types as kernel and user.  I would suggest 
knlink_t and unlink_t but the latter made me want to stab my eyes out 
due to its confusion potential, so I wonder if we should establish a new 
convention with _kt (kernel type) and _ut (user type) suffixes, so 
nlink_kt and nlink_ut, alternatively one could consider k_nlink_t and 
u_nlink_t.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: new architectures, time_t __kernel_long_t

[Al Viro]

On Thu, Dec 20, 2012 at 09:05:30PM -0800, H. Peter Anvin wrote:
> >... as long as we do not have typedef __kernel_foo_t foo_t in linux/types.h.
> >
> 
> In the case of things like nlink_t and dev_t I would suggest we
> explicitly call out the types as kernel and user.  I would suggest
> knlink_t and unlink_t but the latter made me want to stab my eyes
> out due to its confusion potential, so I wonder if we should
> establish a new convention with _kt (kernel type) and _ut (user
> type) suffixes, so nlink_kt and nlink_ut, alternatively one could
> consider k_nlink_t and u_nlink_t.

What the hell for?  _Which_ userland nlink_t, anyway?  We have quite a few
struct stat variants in there.  Sorry, but I really don't see any point
in that, and a lot of potential for confusion.  Marshalling is about the
only thing we need the userland ones for and the code doing it is just fine
with the only object of that type in sight being the field of e.g. struct stat
with given name...

Re: new architectures, time_t __kernel_long_t

[H. Peter Anvin]

Fair enough.

Al Viro <viro@ZenIV.linux.org.uk> wrote:

>On Thu, Dec 20, 2012 at 09:05:30PM -0800, H. Peter Anvin wrote:
>> >... as long as we do not have typedef __kernel_foo_t foo_t in
>linux/types.h.
>> >
>> 
>> In the case of things like nlink_t and dev_t I would suggest we
>> explicitly call out the types as kernel and user.  I would suggest
>> knlink_t and unlink_t but the latter made me want to stab my eyes
>> out due to its confusion potential, so I wonder if we should
>> establish a new convention with _kt (kernel type) and _ut (user
>> type) suffixes, so nlink_kt and nlink_ut, alternatively one could
>> consider k_nlink_t and u_nlink_t.
>
>What the hell for?  _Which_ userland nlink_t, anyway?  We have quite a
>few
>struct stat variants in there.  Sorry, but I really don't see any point
>in that, and a lot of potential for confusion.  Marshalling is about
>the
>only thing we need the userland ones for and the code doing it is just
>fine
>with the only object of that type in sight being the field of e.g.
>struct stat
>with given name...

-- 
Sent from my mobile phone. Please excuse brevity and lack of formatting.