[Xenomai] Xenomai 3: comapt issues with RTDM

[Xenomai] Xenomai 3: comapt issues with RTDM

[Jan Kiszka]

Hi Philippe,

just a heads up, I'll try to address this later:

I received a report that include/rtdm/uapi/rtdm.h contains some data
structures that are used between the kernel and the userspace library
(so not directly by applications) and are not compatible with
32-on-64-bit (compat) scenarios. We should probably promote anything
that is longer on 64-bit to that size, unconditionally. As this affects
the ABI, it should be fixed before 3.0 release, ideally.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/21/2015 02:27 PM, Jan Kiszka wrote:
> Hi Philippe,
> 
> just a heads up, I'll try to address this later:
> 
> I received a report that include/rtdm/uapi/rtdm.h contains some data
> structures that are used between the kernel and the userspace library
> (so not directly by applications) and are not compatible with
> 32-on-64-bit (compat) scenarios. We should probably promote anything
> that is longer on 64-bit to that size, unconditionally. As this affects
> the ABI, it should be fixed before 3.0 release, ideally.
> 

That would require to fix up the client drivers (setsockopt, getsockopt
typically). Ok, I'll have a look when I'm done with testing the blackfin
port.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Jan Kiszka]

On 2015-07-21 14:51, Philippe Gerum wrote:
> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>> Hi Philippe,
>>
>> just a heads up, I'll try to address this later:
>>
>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>> structures that are used between the kernel and the userspace library
>> (so not directly by applications) and are not compatible with
>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>> that is longer on 64-bit to that size, unconditionally. As this affects
>> the ABI, it should be fixed before 3.0 release, ideally.
>>
> 
> That would require to fix up the client drivers (setsockopt, getsockopt
> typically). Ok, I'll have a look when I'm done with testing the blackfin
> port.

Indeed. Just realized that there are already compat structs for the
socket stuff. However, and that was what our user stumbled over, there
is none for _rtdm_mmap_request. But that should be really internal, right?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/21/2015 04:54 PM, Jan Kiszka wrote:
> On 2015-07-21 14:51, Philippe Gerum wrote:
>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>> Hi Philippe,
>>>
>>> just a heads up, I'll try to address this later:
>>>
>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>> structures that are used between the kernel and the userspace library
>>> (so not directly by applications) and are not compatible with
>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>
>>
>> That would require to fix up the client drivers (setsockopt, getsockopt
>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>> port.
> 
> Indeed. Just realized that there are already compat structs for the
> socket stuff. 

Yes, used by the RTIPC support which has been ported already.

> However, and that was what our user stumbled over, there
> is none for _rtdm_mmap_request. But that should be really internal, right?
> 

Yes, we'd only need some thunk between lib/cobalt and
COBALT_SYSCALL(mmap) for this type.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/21/2015 04:54 PM, Jan Kiszka wrote:
> On 2015-07-21 14:51, Philippe Gerum wrote:
>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>> Hi Philippe,
>>>
>>> just a heads up, I'll try to address this later:
>>>
>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>> structures that are used between the kernel and the userspace library
>>> (so not directly by applications) and are not compatible with
>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>
>>
>> That would require to fix up the client drivers (setsockopt, getsockopt
>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>> port.
> 
> Indeed. Just realized that there are already compat structs for the
> socket stuff. However, and that was what our user stumbled over, there
> is none for _rtdm_mmap_request. But that should be really internal, right?
> 

mmap() has the compat thunk for 32<->64 bit conversion already, so this
is ok.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Jan Kiszka]

On 2015-07-23 10:40, Philippe Gerum wrote:
> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>> Hi Philippe,
>>>>
>>>> just a heads up, I'll try to address this later:
>>>>
>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>> structures that are used between the kernel and the userspace library
>>>> (so not directly by applications) and are not compatible with
>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>
>>>
>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>> port.
>>
>> Indeed. Just realized that there are already compat structs for the
>> socket stuff. However, and that was what our user stumbled over, there
>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>
> 
> mmap() has the compat thunk for 32<->64 bit conversion already, so this
> is ok.

Ok... Rechecking the bug report here, the issue is rather the case that
off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
userland. We are apparently required to account for that case as well.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/23/2015 11:08 AM, Jan Kiszka wrote:
> On 2015-07-23 10:40, Philippe Gerum wrote:
>> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>>> Hi Philippe,
>>>>>
>>>>> just a heads up, I'll try to address this later:
>>>>>
>>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>>> structures that are used between the kernel and the userspace library
>>>>> (so not directly by applications) and are not compatible with
>>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>>
>>>>
>>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>>> port.
>>>
>>> Indeed. Just realized that there are already compat structs for the
>>> socket stuff. However, and that was what our user stumbled over, there
>>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>>
>>
>> mmap() has the compat thunk for 32<->64 bit conversion already, so this
>> is ok.
> 
> Ok... Rechecking the bug report here, the issue is rather the case that
> off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
> userland. We are apparently required to account for that case as well.
> 

Ok, so let's move the offset member to loff_t and compat_loff_t in the
mmap request descriptors, that should do the trick.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Jan Kiszka]

On 2015-07-23 11:22, Philippe Gerum wrote:
> On 07/23/2015 11:08 AM, Jan Kiszka wrote:
>> On 2015-07-23 10:40, Philippe Gerum wrote:
>>> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>>>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>>>> Hi Philippe,
>>>>>>
>>>>>> just a heads up, I'll try to address this later:
>>>>>>
>>>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>>>> structures that are used between the kernel and the userspace library
>>>>>> (so not directly by applications) and are not compatible with
>>>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>>>
>>>>>
>>>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>>>> port.
>>>>
>>>> Indeed. Just realized that there are already compat structs for the
>>>> socket stuff. However, and that was what our user stumbled over, there
>>>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>>>
>>>
>>> mmap() has the compat thunk for 32<->64 bit conversion already, so this
>>> is ok.
>>
>> Ok... Rechecking the bug report here, the issue is rather the case that
>> off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
>> userland. We are apparently required to account for that case as well.
>>
> 
> Ok, so let's move the offset member to loff_t and compat_loff_t in the
> mmap request descriptors, that should do the trick.
> 

I only forwarded this suggestion internally, unfortunately cannot tell
yet how it was implemented, just received the early feedback that it is
not yet working. Maybe we are overseeing something, maybe something went
wrong with the implementation. Will be checked tomorrow.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/23/2015 03:37 PM, Jan Kiszka wrote:
> On 2015-07-23 11:22, Philippe Gerum wrote:
>> On 07/23/2015 11:08 AM, Jan Kiszka wrote:
>>> On 2015-07-23 10:40, Philippe Gerum wrote:
>>>> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>>>>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>>>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>>>>> Hi Philippe,
>>>>>>>
>>>>>>> just a heads up, I'll try to address this later:
>>>>>>>
>>>>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>>>>> structures that are used between the kernel and the userspace library
>>>>>>> (so not directly by applications) and are not compatible with
>>>>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>>>>
>>>>>>
>>>>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>>>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>>>>> port.
>>>>>
>>>>> Indeed. Just realized that there are already compat structs for the
>>>>> socket stuff. However, and that was what our user stumbled over, there
>>>>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>>>>
>>>>
>>>> mmap() has the compat thunk for 32<->64 bit conversion already, so this
>>>> is ok.
>>>
>>> Ok... Rechecking the bug report here, the issue is rather the case that
>>> off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
>>> userland. We are apparently required to account for that case as well.
>>>
>>
>> Ok, so let's move the offset member to loff_t and compat_loff_t in the
>> mmap request descriptors, that should do the trick.
>>
> 
> I only forwarded this suggestion internally, unfortunately cannot tell
> yet how it was implemented, just received the early feedback that it is
> not yet working. Maybe we are overseeing something, maybe something went
> wrong with the implementation. Will be checked tomorrow.
> 

This is likely an issue with the native alignment of those types, the
kernel may offset long members in the compat struct based on different
constraints than -m32 does.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/23/2015 09:45 PM, Philippe Gerum wrote:
> On 07/23/2015 03:37 PM, Jan Kiszka wrote:
>> On 2015-07-23 11:22, Philippe Gerum wrote:
>>> On 07/23/2015 11:08 AM, Jan Kiszka wrote:
>>>> On 2015-07-23 10:40, Philippe Gerum wrote:
>>>>> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>>>>>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>>>>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>>>>>> Hi Philippe,
>>>>>>>>
>>>>>>>> just a heads up, I'll try to address this later:
>>>>>>>>
>>>>>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>>>>>> structures that are used between the kernel and the userspace library
>>>>>>>> (so not directly by applications) and are not compatible with
>>>>>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>>>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>>>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>>>>>
>>>>>>>
>>>>>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>>>>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>>>>>> port.
>>>>>>
>>>>>> Indeed. Just realized that there are already compat structs for the
>>>>>> socket stuff. However, and that was what our user stumbled over, there
>>>>>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>>>>>
>>>>>
>>>>> mmap() has the compat thunk for 32<->64 bit conversion already, so this
>>>>> is ok.
>>>>
>>>> Ok... Rechecking the bug report here, the issue is rather the case that
>>>> off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
>>>> userland. We are apparently required to account for that case as well.
>>>>
>>>
>>> Ok, so let's move the offset member to loff_t and compat_loff_t in the
>>> mmap request descriptors, that should do the trick.
>>>
>>
>> I only forwarded this suggestion internally, unfortunately cannot tell
>> yet how it was implemented, just received the early feedback that it is
>> not yet working. Maybe we are overseeing something, maybe something went
>> wrong with the implementation. Will be checked tomorrow.
>>
> 
> This is likely an issue with the native alignment of those types, the
> kernel may offset long members in the compat struct based on different
> constraints than -m32 does.
> 

And we probably also need the mmap64 wrapper we don't have yet. Ok, I'm
working on this.

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Philippe Gerum]

On 07/24/2015 09:24 AM, Philippe Gerum wrote:
> On 07/23/2015 09:45 PM, Philippe Gerum wrote:
>> On 07/23/2015 03:37 PM, Jan Kiszka wrote:
>>> On 2015-07-23 11:22, Philippe Gerum wrote:
>>>> On 07/23/2015 11:08 AM, Jan Kiszka wrote:
>>>>> On 2015-07-23 10:40, Philippe Gerum wrote:
>>>>>> On 07/21/2015 04:54 PM, Jan Kiszka wrote:
>>>>>>> On 2015-07-21 14:51, Philippe Gerum wrote:
>>>>>>>> On 07/21/2015 02:27 PM, Jan Kiszka wrote:
>>>>>>>>> Hi Philippe,
>>>>>>>>>
>>>>>>>>> just a heads up, I'll try to address this later:
>>>>>>>>>
>>>>>>>>> I received a report that include/rtdm/uapi/rtdm.h contains some data
>>>>>>>>> structures that are used between the kernel and the userspace library
>>>>>>>>> (so not directly by applications) and are not compatible with
>>>>>>>>> 32-on-64-bit (compat) scenarios. We should probably promote anything
>>>>>>>>> that is longer on 64-bit to that size, unconditionally. As this affects
>>>>>>>>> the ABI, it should be fixed before 3.0 release, ideally.
>>>>>>>>>
>>>>>>>>
>>>>>>>> That would require to fix up the client drivers (setsockopt, getsockopt
>>>>>>>> typically). Ok, I'll have a look when I'm done with testing the blackfin
>>>>>>>> port.
>>>>>>>
>>>>>>> Indeed. Just realized that there are already compat structs for the
>>>>>>> socket stuff. However, and that was what our user stumbled over, there
>>>>>>> is none for _rtdm_mmap_request. But that should be really internal, right?
>>>>>>>
>>>>>>
>>>>>> mmap() has the compat thunk for 32<->64 bit conversion already, so this
>>>>>> is ok.
>>>>>
>>>>> Ok... Rechecking the bug report here, the issue is rather the case that
>>>>> off_t becomes 64 bit when defining -D_FILE_OFFSET_BITS=64 for your
>>>>> userland. We are apparently required to account for that case as well.
>>>>>
>>>>
>>>> Ok, so let's move the offset member to loff_t and compat_loff_t in the
>>>> mmap request descriptors, that should do the trick.
>>>>
>>>
>>> I only forwarded this suggestion internally, unfortunately cannot tell
>>> yet how it was implemented, just received the early feedback that it is
>>> not yet working. Maybe we are overseeing something, maybe something went
>>> wrong with the implementation. Will be checked tomorrow.
>>>
>>
>> This is likely an issue with the native alignment of those types, the
>> kernel may offset long members in the compat struct based on different
>> constraints than -m32 does.
>>
> 
> And we probably also need the mmap64 wrapper we don't have yet. Ok, I'm
> working on this.
> 

Pushed to -next, and tested over 32 and 3264 ABI models over x86. This
breaks the ABI, so both kernel and userland have to be rebuilt. The bad
news is that I now have to redo all the testing on all other archs. Oh
well, ...

-- 
Philippe.

Re: [Xenomai] Xenomai 3: comapt issues with RTDM

[Jan Kiszka]

On 2015-07-24 12:02, Philippe Gerum wrote:
> Pushed to -next, and tested over 32 and 3264 ABI models over x86. This
> breaks the ABI, so both kernel and userland have to be rebuilt. The bad
> news is that I now have to redo all the testing on all other archs. Oh
> well, ...

Was tested here (except for the new *64 calls) and works well, thanks!

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux