[Draft RFC] Bring POSIX signal to evl

[Draft RFC] Bring POSIX signal to evl

[Qichen Qiu]

Hi all, 

Currently, EVL does not support POSIX signals. To better align with 
Xenomai and Cobalt (see the CXP [1]), we need to add POSIX signal 
support to EVL. As far as I can tell, there may be two potential 
approaches: 

# Approach 1: Adapt the current flags mechanism to POSIX signals 

EVL offers an alternative to signals called flags [1], which allows 
sending a 32-bit word to another thread. The flags mainly offers the
service:
* use `evl_create_flags` or `evl_open_flags` to open a flag element. 
* use `evl_wait_flags` to wait a specific bit and evl_post_flags to 
post one or some bits. 

The mechanism is like a POSIX real-time signal without queueing. 
However, there are notable differences between flags and POSIX signals 
that would need to be addressed:
* Signal Capacity: Cobalt supports 63 signals (excluding pseudo 
signals), while flags are limited to 32 due to the monitor's design. 

* Initialization: POSIX signals are pre-configured and require no 
explicit initialization by the sender or receiver. In contrast, EVL's 
flags require users to initialize and open them manually. 

* Target: In POSIX, signals can target either processes or threads, 
whereas in EVL, the target mechanism is more limited. 

* Sigqueue: POSIX signals can carry additional data (sigqueue), while 
EVL flags are restricted to 4 bytes of payload. 

* Queueing: POSIX real-time signals are queued, whereas EVL flags only 
store the most recent signal, making them more similar to standard 
signals. 

While it is possible to adapt the current flags implementation to 
address these differences, I believe doing so would overcomplicate the 
existing mechanism. For this reason, I prefer an alternative approach: 
migrating the Xenomai POSIX signal subsystem to EVL. 

# Approach 2: Migrate the Xenomai POSIX Signal Subsystem to EVL 

Xenomai implements a signal subsystem. For each Cobalt thread, it 
maintains:
* A wait channel (sigwait) to sleep and wake up while waiting for a 
signal. 
* A sigqueue structure to store pending signals. 
* A sigwait_context to hold the user-space waiting context. 
* A field sigpending to track the current pending signal for the thread.

Philippe once suggested that the new POSIX-like API "would be 
implemented as a separate driver using the EVL kernel API" [3], meaning
the POSIX signal support would be loaded as an EVL driver. However, 
since signal sending and receiving are per-thread or per-process 
operations, the signal module would need to attach data to each EVL 
thread. This integration seems challenging for a standalone driver. 
Instead, I propose integrating the implementation directly into the 
current kernel. 

Before starting work on this, I would like to gather feedback and 
suggestions from the community. Thank you for your input. 

Best regards,
Sam 


[1] https://lore.kernel.org/xenomai/87mtyr1abi.fsf@xenomai.org/
[2] https://evlproject.org/core/user-api/flags
[3] https://riot.im/app/#/room/#evlproject:matrix.org/$kFRgs7y48Qr0033GNh-sDnI7GUrrp_mo-OEhDI0GAWk

Re: [Draft RFC] Bring POSIX signal to evl

[Jan Kiszka]

On 18.12.24 13:26, Qichen Qiu wrote:
> Hi all, 
> 
> Currently, EVL does not support POSIX signals. To better align with 
> Xenomai and Cobalt (see the CXP [1]), we need to add POSIX signal 
> support to EVL. As far as I can tell, there may be two potential 
> approaches: 
> 
> # Approach 1: Adapt the current flags mechanism to POSIX signals 
> 
> EVL offers an alternative to signals called flags [1], which allows 
> sending a 32-bit word to another thread. The flags mainly offers the
> service:
> * use `evl_create_flags` or `evl_open_flags` to open a flag element. 
> * use `evl_wait_flags` to wait a specific bit and evl_post_flags to 
> post one or some bits. 
> 
> The mechanism is like a POSIX real-time signal without queueing. 
> However, there are notable differences between flags and POSIX signals 
> that would need to be addressed:
> * Signal Capacity: Cobalt supports 63 signals (excluding pseudo 
> signals), while flags are limited to 32 due to the monitor's design. 
> 
> * Initialization: POSIX signals are pre-configured and require no 
> explicit initialization by the sender or receiver. In contrast, EVL's 
> flags require users to initialize and open them manually. 
> 
> * Target: In POSIX, signals can target either processes or threads, 
> whereas in EVL, the target mechanism is more limited. 
> 
> * Sigqueue: POSIX signals can carry additional data (sigqueue), while 
> EVL flags are restricted to 4 bytes of payload. 
> 
> * Queueing: POSIX real-time signals are queued, whereas EVL flags only 
> store the most recent signal, making them more similar to standard 
> signals. 
> 
> While it is possible to adapt the current flags implementation to 
> address these differences, I believe doing so would overcomplicate the 
> existing mechanism. For this reason, I prefer an alternative approach: 
> migrating the Xenomai POSIX signal subsystem to EVL. 
> 
> # Approach 2: Migrate the Xenomai POSIX Signal Subsystem to EVL 
> 
> Xenomai implements a signal subsystem. For each Cobalt thread, it 
> maintains:
> * A wait channel (sigwait) to sleep and wake up while waiting for a 
> signal. 
> * A sigqueue structure to store pending signals. 
> * A sigwait_context to hold the user-space waiting context. 
> * A field sigpending to track the current pending signal for the thread.
> 
> Philippe once suggested that the new POSIX-like API "would be 
> implemented as a separate driver using the EVL kernel API" [3], meaning
> the POSIX signal support would be loaded as an EVL driver. However, 
> since signal sending and receiving are per-thread or per-process 
> operations, the signal module would need to attach data to each EVL 
> thread. This integration seems challenging for a standalone driver. 
> Instead, I propose integrating the implementation directly into the 
> current kernel. 
> 
> Before starting work on this, I would like to gather feedback and 
> suggestions from the community. Thank you for your input. 

Going for a careful extension, even of core data structures, sounds to
me like the best approach as well. Just also keep on eye on the ongoing
efforts to use posix signals to deliver synchronous fault events to the
causing thread in RT. That we would love to have with EVL one day, too.
It should widely be orthogonal to synchronous waiting on signals, but
better double-check.

Jan

> 
> Best regards,
> Sam 
> 
> 
> [1] https://lore.kernel.org/xenomai/87mtyr1abi.fsf@xenomai.org/
> [2] https://evlproject.org/core/user-api/flags
> [3] https://riot.im/app/#/room/#evlproject:matrix.org/$kFRgs7y48Qr0033GNh-sDnI7GUrrp_mo-OEhDI0GAWk
> 

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

Re: [Draft RFC] Bring POSIX signal to evl

[Philippe Gerum]

Qichen Qiu <ruiqurm@gmail.com> writes:

> Hi all, 
>
> Currently, EVL does not support POSIX signals. To better align with 
> Xenomai and Cobalt (see the CXP [1]), we need to add POSIX signal 
> support to EVL. As far as I can tell, there may be two potential 
> approaches: 
>
> # Approach 1: Adapt the current flags mechanism to POSIX signals 
>
> EVL offers an alternative to signals called flags [1], which allows 
> sending a 32-bit word to another thread. The flags mainly offers the
> service:
> * use `evl_create_flags` or `evl_open_flags` to open a flag element. 
> * use `evl_wait_flags` to wait a specific bit and evl_post_flags to 
> post one or some bits. 
>
> The mechanism is like a POSIX real-time signal without queueing. 
> However, there are notable differences between flags and POSIX signals 
> that would need to be addressed:
> * Signal Capacity: Cobalt supports 63 signals (excluding pseudo 
> signals), while flags are limited to 32 due to the monitor's design. 
>
> * Initialization: POSIX signals are pre-configured and require no 
> explicit initialization by the sender or receiver. In contrast, EVL's 
> flags require users to initialize and open them manually. 
>
> * Target: In POSIX, signals can target either processes or threads, 
> whereas in EVL, the target mechanism is more limited. 
>

Not really. The Cobalt implementation does not support process-oriented
delivery either, only a limited notion of thread groups waiting for a
given signal. x3 signals are fundamentally thread-directed.

> * Sigqueue: POSIX signals can carry additional data (sigqueue), while 
> EVL flags are restricted to 4 bytes of payload. 
>
> * Queueing: POSIX real-time signals are queued, whereas EVL flags only 
> store the most recent signal, making them more similar to standard 
> signals. 
>
> While it is possible to adapt the current flags implementation to 
> address these differences, I believe doing so would overcomplicate the 
> existing mechanism. For this reason, I prefer an alternative approach: 
> migrating the Xenomai POSIX signal subsystem to EVL. 
>

The semantics of the EVL flags way too distinct from the POSIX signals,
so yeah, there is no point in trying to bend the implementation, that
would not work for basic reasons.

> # Approach 2: Migrate the Xenomai POSIX Signal Subsystem to EVL 
>
> Xenomai implements a signal subsystem. For each Cobalt thread, it 
> maintains:
> * A wait channel (sigwait) to sleep and wake up while waiting for a 
> signal. 
> * A sigqueue structure to store pending signals. 
> * A sigwait_context to hold the user-space waiting context. 
> * A field sigpending to track the current pending signal for the thread.
>

I definitely agree that the current signal implementation in x3 should
be reused as much as possible, no need to reinvent that wheel, although
as Jan mentioned already, we'd need to consider async delivery as well,
which is not supported ATM. However, big CAVEAT: the EVL/x4 locking
model is fine-grained like the one the regular kernel implements, unlike
x3 which exhibits a single coarse 'superlock'. Porting will require to
reconsider a significant portion of the original code under this new
light.

> Philippe once suggested that the new POSIX-like API "would be 
> implemented as a separate driver using the EVL kernel API" [3], meaning
> the POSIX signal support would be loaded as an EVL driver. However, 
> since signal sending and receiving are per-thread or per-process 
> operations, the signal module would need to attach data to each EVL 
> thread. This integration seems challenging for a standalone driver. 
> Instead, I propose integrating the implementation directly into the 
> current kernel.

There may be a better approach, keeping the POSIX specifics out of the
EVL core, which was the fundamental reason behind the rewrite btw. I'm
very concerned about bloating the core with POSIX specific stuff, or any
other API flavor for that matter.

A separate driver can implement POSIX support, providing one or more
so-called EVL "elements", accessible as vnodes under /dev/evl, i.e. the
entry point to so-called "factories".

The attachment you mention is only a matter of opening a multiplex
device created by such driver in the /dev/evl hierarchy when attaching a
thread, see evl_create_element() in libevl. So basically, that would
boil down to something like:

__thread __attribute__ ((tls_model (EVL_TLS_MODEL)))
int __evl_posix_efd = -1;

int __evl_attach_posix()
{
        struct evl_element_ids eids;
       
        ...
        __evl_posix_efd = evl_create_element("posix", "signals",
                NULL, EVL_CLONE_PUBLIC, &eids);
        ...
}

int evl_attach_thread(...)
{
        __evl_attach_posix();
}

Next, process-wide init chores could be called from evl_init() if the
POSIX feature is enabled. 

Then you would use __evl_posix_efd to oob_read/write/ioctl requests
to/from the current thread. The VFS would do all the magic about
resource management, this is a plain regular fd. I can and will
elaborate on this later on as we devise the final scheme.

Bottom-line: some changes to the EVL core may be needed to facilitate
the integration of POSIX, but I'm pretty sure this could be made by
adding generic helpers, instead of dumping yet another API directly into
a fairly streamlined core.

-- 
Philippe.

Re: [Draft RFC] Bring POSIX signal to evl

[Qichen Qiu]

> Just also keep on eye on the ongoing efforts to use posix signals to
> deliver synchronous fault events to the causing thread in RT.

Yes. I could have a look at the implementation of vanilla Linux signal
and see what I can do


Jan Kiszka <jan.kiszka@siemens.com> 于2024年12月19日周四 22:31写道：
>
> On 18.12.24 13:26, Qichen Qiu wrote:
> > Hi all,
> >
> > Currently, EVL does not support POSIX signals. To better align with
> > Xenomai and Cobalt (see the CXP [1]), we need to add POSIX signal
> > support to EVL. As far as I can tell, there may be two potential
> > approaches:
> >
> > # Approach 1: Adapt the current flags mechanism to POSIX signals
> >
> > EVL offers an alternative to signals called flags [1], which allows
> > sending a 32-bit word to another thread. The flags mainly offers the
> > service:
> > * use `evl_create_flags` or `evl_open_flags` to open a flag element.
> > * use `evl_wait_flags` to wait a specific bit and evl_post_flags to
> > post one or some bits.
> >
> > The mechanism is like a POSIX real-time signal without queueing.
> > However, there are notable differences between flags and POSIX signals
> > that would need to be addressed:
> > * Signal Capacity: Cobalt supports 63 signals (excluding pseudo
> > signals), while flags are limited to 32 due to the monitor's design.
> >
> > * Initialization: POSIX signals are pre-configured and require no
> > explicit initialization by the sender or receiver. In contrast, EVL's
> > flags require users to initialize and open them manually.
> >
> > * Target: In POSIX, signals can target either processes or threads,
> > whereas in EVL, the target mechanism is more limited.
> >
> > * Sigqueue: POSIX signals can carry additional data (sigqueue), while
> > EVL flags are restricted to 4 bytes of payload.
> >
> > * Queueing: POSIX real-time signals are queued, whereas EVL flags only
> > store the most recent signal, making them more similar to standard
> > signals.
> >
> > While it is possible to adapt the current flags implementation to
> > address these differences, I believe doing so would overcomplicate the
> > existing mechanism. For this reason, I prefer an alternative approach:
> > migrating the Xenomai POSIX signal subsystem to EVL.
> >
> > # Approach 2: Migrate the Xenomai POSIX Signal Subsystem to EVL
> >
> > Xenomai implements a signal subsystem. For each Cobalt thread, it
> > maintains:
> > * A wait channel (sigwait) to sleep and wake up while waiting for a
> > signal.
> > * A sigqueue structure to store pending signals.
> > * A sigwait_context to hold the user-space waiting context.
> > * A field sigpending to track the current pending signal for the thread.
> >
> > Philippe once suggested that the new POSIX-like API "would be
> > implemented as a separate driver using the EVL kernel API" [3], meaning
> > the POSIX signal support would be loaded as an EVL driver. However,
> > since signal sending and receiving are per-thread or per-process
> > operations, the signal module would need to attach data to each EVL
> > thread. This integration seems challenging for a standalone driver.
> > Instead, I propose integrating the implementation directly into the
> > current kernel.
> >
> > Before starting work on this, I would like to gather feedback and
> > suggestions from the community. Thank you for your input.
>
> Going for a careful extension, even of core data structures, sounds to
> me like the best approach as well. Just also keep on eye on the ongoing
> efforts to use posix signals to deliver synchronous fault events to the
> causing thread in RT. That we would love to have with EVL one day, too.
> It should widely be orthogonal to synchronous waiting on signals, but
> better double-check.
>
> Jan
>
> >
> > Best regards,
> > Sam
> >
> >
> > [1] https://lore.kernel.org/xenomai/87mtyr1abi.fsf@xenomai.org/
> > [2] https://evlproject.org/core/user-api/flags
> > [3] https://riot.im/app/#/room/#evlproject:matrix.org/$kFRgs7y48Qr0033GNh-sDnI7GUrrp_mo-OEhDI0GAWk
> >
>
> --
> Siemens AG, Foundational Technologies
> Linux Expert Center

Re: [Draft RFC] Bring POSIX signal to evl

[Qichen Qiu]

Hi.
Thanks for your suggestions. It is really helpful.

> However, big CAVEAT: the EVL/x4 locking model is fine-grained like the
> one the regular kernel implements, unlike x3 which exhibits a single
> coarse 'superlock'. Porting will require to reconsider a significant
> portion of the original code under this new light.

Do you mean the `nklock` that is used in almost every syscall? I wonder
what the original purpose of this lock is. Is that a global lock like the BKL?


Philippe Gerum <rpm@xenomai.org> 于2024年12月19日周四 23:43写道：
>
> Qichen Qiu <ruiqurm@gmail.com> writes:
>
> > Hi all,
> >
> > Currently, EVL does not support POSIX signals. To better align with
> > Xenomai and Cobalt (see the CXP [1]), we need to add POSIX signal
> > support to EVL. As far as I can tell, there may be two potential
> > approaches:
> >
> > # Approach 1: Adapt the current flags mechanism to POSIX signals
> >
> > EVL offers an alternative to signals called flags [1], which allows
> > sending a 32-bit word to another thread. The flags mainly offers the
> > service:
> > * use `evl_create_flags` or `evl_open_flags` to open a flag element.
> > * use `evl_wait_flags` to wait a specific bit and evl_post_flags to
> > post one or some bits.
> >
> > The mechanism is like a POSIX real-time signal without queueing.
> > However, there are notable differences between flags and POSIX signals
> > that would need to be addressed:
> > * Signal Capacity: Cobalt supports 63 signals (excluding pseudo
> > signals), while flags are limited to 32 due to the monitor's design.
> >
> > * Initialization: POSIX signals are pre-configured and require no
> > explicit initialization by the sender or receiver. In contrast, EVL's
> > flags require users to initialize and open them manually.
> >
> > * Target: In POSIX, signals can target either processes or threads,
> > whereas in EVL, the target mechanism is more limited.
> >
>
> Not really. The Cobalt implementation does not support process-oriented
> delivery either, only a limited notion of thread groups waiting for a
> given signal. x3 signals are fundamentally thread-directed.
>
> > * Sigqueue: POSIX signals can carry additional data (sigqueue), while
> > EVL flags are restricted to 4 bytes of payload.
> >
> > * Queueing: POSIX real-time signals are queued, whereas EVL flags only
> > store the most recent signal, making them more similar to standard
> > signals.
> >
> > While it is possible to adapt the current flags implementation to
> > address these differences, I believe doing so would overcomplicate the
> > existing mechanism. For this reason, I prefer an alternative approach:
> > migrating the Xenomai POSIX signal subsystem to EVL.
> >
>
> The semantics of the EVL flags way too distinct from the POSIX signals,
> so yeah, there is no point in trying to bend the implementation, that
> would not work for basic reasons.
>
> > # Approach 2: Migrate the Xenomai POSIX Signal Subsystem to EVL
> >
> > Xenomai implements a signal subsystem. For each Cobalt thread, it
> > maintains:
> > * A wait channel (sigwait) to sleep and wake up while waiting for a
> > signal.
> > * A sigqueue structure to store pending signals.
> > * A sigwait_context to hold the user-space waiting context.
> > * A field sigpending to track the current pending signal for the thread.
> >
>
> I definitely agree that the current signal implementation in x3 should
> be reused as much as possible, no need to reinvent that wheel, although
> as Jan mentioned already, we'd need to consider async delivery as well,
> which is not supported ATM. However, big CAVEAT: the EVL/x4 locking
> model is fine-grained like the one the regular kernel implements, unlike
> x3 which exhibits a single coarse 'superlock'. Porting will require to
> reconsider a significant portion of the original code under this new
> light.
>
> > Philippe once suggested that the new POSIX-like API "would be
> > implemented as a separate driver using the EVL kernel API" [3], meaning
> > the POSIX signal support would be loaded as an EVL driver. However,
> > since signal sending and receiving are per-thread or per-process
> > operations, the signal module would need to attach data to each EVL
> > thread. This integration seems challenging for a standalone driver.
> > Instead, I propose integrating the implementation directly into the
> > current kernel.
>
> There may be a better approach, keeping the POSIX specifics out of the
> EVL core, which was the fundamental reason behind the rewrite btw. I'm
> very concerned about bloating the core with POSIX specific stuff, or any
> other API flavor for that matter.
>
> A separate driver can implement POSIX support, providing one or more
> so-called EVL "elements", accessible as vnodes under /dev/evl, i.e. the
> entry point to so-called "factories".
>
> The attachment you mention is only a matter of opening a multiplex
> device created by such driver in the /dev/evl hierarchy when attaching a
> thread, see evl_create_element() in libevl. So basically, that would
> boil down to something like:
>
> __thread __attribute__ ((tls_model (EVL_TLS_MODEL)))
> int __evl_posix_efd = -1;
>
> int __evl_attach_posix()
> {
>         struct evl_element_ids eids;
>
>         ...
>         __evl_posix_efd = evl_create_element("posix", "signals",
>                 NULL, EVL_CLONE_PUBLIC, &eids);
>         ...
> }
>
> int evl_attach_thread(...)
> {
>         __evl_attach_posix();
> }
>
> Next, process-wide init chores could be called from evl_init() if the
> POSIX feature is enabled.
>
> Then you would use __evl_posix_efd to oob_read/write/ioctl requests
> to/from the current thread. The VFS would do all the magic about
> resource management, this is a plain regular fd. I can and will
> elaborate on this later on as we devise the final scheme.
>
> Bottom-line: some changes to the EVL core may be needed to facilitate
> the integration of POSIX, but I'm pretty sure this could be made by
> adding generic helpers, instead of dumping yet another API directly into
> a fairly streamlined core.
>
> --
> Philippe.

Re: [Draft RFC] Bring POSIX signal to evl

[Philippe Gerum]

Qichen Qiu <ruiqurm@gmail.com> writes:

> Hi.
> Thanks for your suggestions. It is really helpful.
>
>> However, big CAVEAT: the EVL/x4 locking model is fine-grained like the
>> one the regular kernel implements, unlike x3 which exhibits a single
>> coarse 'superlock'. Porting will require to reconsider a significant
>> portion of the original code under this new light.
>
> Do you mean the `nklock` that is used in almost every syscall? I wonder
> what the original purpose of this lock is. Is that a global lock like the BKL?
>

Yes, this is a recursive single lock coming from the Dark Ages, which is
basically used throughout the core to serialize access to almost all
resources. This is why x3 does not scale that well SMP-wise.

-- 
Philippe.