[Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[hauptmech]

I've used and followed Xenomai off and on since it forked from RTAI. I'm 
building a new xenomai x86 system after a few years of doing other 
things. I'm noticing that the adeos patches are few and relatively old 
compared to the kernel for the x86 architecture. Kernel version 3.2 has 
moved from .21 to .35 for instance.

So my question is (with the deepest respect for the effort it must take) 
why?

Is

[Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[hauptmech]

(apologies for the previous incomplete email)

I've used and followed Xenomai off and on since it forked from RTAI. I'm 
building a new xenomai x86 system after a few years of doing other 
things. I'm noticing that the adeos patches are few and relatively old 
compared to the kernel for the x86 architecture. Kernel version 3.2 has 
moved from .21 to .35 for instance. Meanwhile my (non-rt) systems all 
use 3.4 or 3.6.

So my question is (with the deepest respect for the effort it must take) 
why? Why do the adeos patches lag so much?

Also why were the linux-3.2.21 x86 patch typos which prevent compilation 
not fixed? When such basic thing is not taken care of it makes me doubt 
the integrity of the whole project.

In my community (robotics) xenomai is the go-to RT system for me and 
most of my peers. However the xenomai roadmap and task-market are 
bitrotted and there don't seem to be project status updates on the wiki 
or mailing list which makes it hard to contribute or make plans.

I guess I just want to call attention to the fact that things are 
looking a little rough at xenomai.org and ask whether passers by like 
myself should try to help a little or move on?

kind regards,

hauptmech

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Tim Niemeyer]

Hi

Am Freitag, den 14.12.2012, 15:10 +0100 schrieb hauptmech:
> I've used and followed Xenomai off and on since it forked from RTAI. I'm 
> building a new xenomai x86 system after a few years of doing other 
> things. I'm noticing that the adeos patches are few and relatively old 
> compared to the kernel for the x86 architecture. Kernel version 3.2 has 
> moved from .21 to .35 for instance. Meanwhile my (non-rt) systems all 
> use 3.4 or 3.6.

I use 3.4 on my rt system. You can find the adeos work from the
Xenomai-Team here:
http://git.xenomai.org/?p=ipipe-gch.git;a=shortlog;h=refs/heads/for-core-3.4

It was quite confused to get it working on 3.4.14. As the work is based
on 3.4.6.

There is also a branch 'for-core-3.5'.

-- 
Tim Niemeyer

Corscience GmbH & Co. KG
Henkestr. 91
D-91052 Erlangen
Germany

e-mail: tim.niemeyer@corscience.de
Internet: www.corscience.de

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Philippe Gerum]

On 12/14/2012 02:13 PM, hauptmech wrote:
>
> I've used and followed Xenomai off and on since it forked from RTAI.

It did not. A fork happens when you diverge from a common code base, 
which does not apply to Xenomai wrt RTAI. Both used to be independent 
projects before they joined forces, then split as we failed to actually 
merge both real-time cores. Portions of Adeos might still be shared by 
both projects, but incidentally, it is a Xenomai-originated 
contribution. Please check 
https://www.xenomai.org/index.php/Xenomai:History for facts.

  I'm
> building a new xenomai x86 system after a few years of doing other
> things. I'm noticing that the adeos patches are few and relatively old
> compared to the kernel for the x86 architecture.

We published 990 official Adeos patches since 2001, with more than half 
of them targeting the x86_32 and _64 architectures. I would not call 
these a few. Please check download.gna.org/adeos/patches for more 
information.

  Kernel version 3.2 has
> moved from .21 to .35 for instance.
>

We never updated the Adeos patches for minor linux releases routinely, 
this has never been our policy, and will likely never be the case. This 
would just involve way too much work for our bandwidth. We are focusing 
on major releases. So you seem to be expecting something we never 
delivered in the past anyway.

However, the pace of our Adeos updates for major kernel releases has 
slowed down over the past three years, definitely. This said, we have 
support for kernel 3.4 and in some cases 3.5, for the main architecture 
ports we maintain, it's not lagging that far behind.

> So my question is (with the deepest respect for the effort it must take)
> why?
>

Because we all have to deal with priorities and available resources, 
this is no different for the Xenomai maintainers.

> Is
>
>
>
>
>
> _______________________________________________
> Xenomai mailing list
> Xenomai@xenomai.org
> http://www.xenomai.org/mailman/listinfo/xenomai
>


-- 
Philippe.

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Gilles Chanteperdrix]

On 12/14/2012 11:12 PM, Philippe Gerum wrote:

> On 12/14/2012 02:13 PM, hauptmech wrote:
>>
>> I've used and followed Xenomai off and on since it forked from RTAI.
> 
> It did not. A fork happens when you diverge from a common code base, 
> which does not apply to Xenomai wrt RTAI. Both used to be independent 
> projects before they joined forces, then split as we failed to actually 
> merge both real-time cores. Portions of Adeos might still be shared by 
> both projects, but incidentally, it is a Xenomai-originated 
> contribution. Please check 
> https://www.xenomai.org/index.php/Xenomai:History for facts.
> 
>   I'm
>> building a new xenomai x86 system after a few years of doing other
>> things. I'm noticing that the adeos patches are few and relatively old
>> compared to the kernel for the x86 architecture.
> 
> We published 990 official Adeos patches since 2001, with more than half 
> of them targeting the x86_32 and _64 architectures. I would not call 
> these a few. Please check download.gna.org/adeos/patches for more 
> information.
> 
>   Kernel version 3.2 has
>> moved from .21 to .35 for instance.
>>
> 
> We never updated the Adeos patches for minor linux releases routinely, 
> this has never been our policy, and will likely never be the case. This 
> would just involve way too much work for our bandwidth. We are focusing 
> on major releases. So you seem to be expecting something we never 
> delivered in the past anyway.
> 
> However, the pace of our Adeos updates for major kernel releases has 
> slowed down over the past three years, definitely. This said, we have 
> support for kernel 3.4 and in some cases 3.5, for the main architecture 
> ports we maintain, it's not lagging that far behind.
> 
>> So my question is (with the deepest respect for the effort it must take)
>> why?
>>
> 
> Because we all have to deal with priorities and available resources, 
> this is no different for the Xenomai maintainers.


Speaking for myself here, porting the I-pipe patch (for the ARM
architecture in my case) to new versions of Linux is the most boring
part of the job, and since I do it on my free time, I tend to
procrastinate. Which is why for instance the I-pipe patch for 3.5 has
been available since august, and I did the port to the ARM architecture
only last week-end (shame on me...).

Anyway, I think the real problem for an end user is the frequency of
Xenomai releases, not of the I-pipe patches, because we probably should
have made a release of Xenomai as soon as the patch for 3.4 has been
available on all architectures. But on the other hand, if you look at
the core-3.4 branch in the git repository, you will see that staying
some time on that patch allowed us to fix many issues.

As for the typos which prevent compilation in some cases, the problem
has been the same for years: the maintainers use kernel configurations
optimized for their use case (fast compilation time, good latencies for
one platform) and the users use opposite configuration options (kernels
as generic as possible). As I said several time, I would like, with the
next release of Xenomai, to release a kernel as a Debian package, so, we
would also test a "Debian-like" configuration. Another situation unique
to the I-pipe patch for Linux 3.2 is that it is based on a new architecture.

-- 
                                                                Gilles.

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[hauptmech]

I'm coming to the documentation with relatively fresh eyes. If you give 
me a user on the wiki I'm happy to do a little gardening. Things like 
the [getting started] page which is useful but not linked to anymore. 
Perhaps a buffer page between the wiki and the git page with a little 
description of the repos and how they fit into the development workflow 
(where a link to denx could go).

I have made xenomai source packages for pacman (Archlinux + a few 
others) for arm last year and just now x86. However since archlinux 
tends to use a minimally patched vanilla kernel it's not as exciting as 
debian.

More embedded below...

On 12/15/2012 12:47 PM, Gilles Chanteperdrix wrote:
> On 12/14/2012 11:12 PM, Philippe Gerum wrote:
>
>> We never updated the Adeos patches for minor linux releases routinely,
>> this has never been our policy, and will likely never be the case. This
>> would just involve way too much work for our bandwidth. We are focusing
>> on major releases. So you seem to be expecting something we never
>> delivered in the past anyway.
Ok, I wasn't expecting anything. I just didn't understand.

>>
>> However, the pace of our Adeos updates for major kernel releases has
>> slowed down over the past three years, definitely. This said, we have
>> support for kernel 3.4 and in some cases 3.5, for the main architecture
>> ports we maintain, it's not lagging that far behind.
The availability of ipipe for 3.4/3.5 (x86) is not apparent from the 
website nor the patches in the xenomai source.
>>
>>> So my question is (with the deepest respect for the effort it must take)
>>> why?
>>>
>> Because we all have to deal with priorities and available resources,
>> this is no different for the Xenomai maintainers.
>
> Speaking for myself here, porting the I-pipe patch (for the ARM
> architecture in my case) to new versions of Linux is the most boring
> part of the job, and since I do it on my free time, I tend to
> procrastinate. Which is why for instance the I-pipe patch for 3.5 has
> been available since august, and I did the port to the ARM architecture
> only last week-end (shame on me...).
Does all the work involve expertise with the kernel? I realize that 
having a inexpert 'helper' for a lot of tasks just creates extra work 
but if that's not the case here, perhaps a help-wanted post on the wiki 
and some documentation on the process would attract someone wanting to 
learn or help.
>
> Anyway, I think the real problem for an end user is the frequency of
> Xenomai releases, not of the I-pipe patches, because we probably should
> have made a release of Xenomai as soon as the patch for 3.4 has been
> available on all architectures. But on the other hand, if you look at
> the core-3.4 branch in the git repository, you will see that staying
> some time on that patch allowed us to fix many issues.
I look forward to the release when it does happen.

-hauptmech

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Gilles Chanteperdrix]

On 12/16/2012 02:41 PM, hauptmech wrote:

> 
> I'm coming to the documentation with relatively fresh eyes. If you give 
> me a user on the wiki I'm happy to do a little gardening. Things like 
> the [getting started] page which is useful but not linked to anymore. 
> Perhaps a buffer page between the wiki and the git page with a little 
> description of the repos and how they fit into the development workflow 
> (where a link to denx could go).


The "getting started" page is unfinished (contains empty paragraphs),
and outdated (contains dead links and references to things like the
simulator which is currently not really usable) at the same time. I do
not believe we have the resources to maintain such a page up-to-date.

>From my point of view, what we need is:
- up to date manual pages of the binaries in xenomai distribution, in
the xenomai sources, we can start from the current manual pages, but
some of them are outdated and we should convert them to asciidoc so that
more people can work on them, and we can put the html versions on the
web site (with automatic update, as currently happens for the API
documentation and README.INSTALL/TROUBLESHOOTING)
- some thematic pages on the wiki such as:
. calibrating your system with xeno-test
. influence of the various Linux kernel options on latencies on at least
the arm and x86 platforms
- and hyperlinks, for instance at the end of README.INSTALL point to the
"calibrating your system" page, and in this page, have an hyperlink to
the xeno-test and latency test manual pages, in the page "porting the
I-pipe-core patch to a new ARM board", mention the tools that can be
used to debug the various issues, and have an hyperlink to their manual
page.
- a user guide, the "porting posix applications to xenomai" page, was my
attempt of a user-guide, but it has a very specialized goal, whereas
part of the information it contains is general to anyone writing posix
applications for xenomai, and evern applications in general.
- something we have talked about but has long gone forgotten, have some
kind of database of the latency results sortable by a mix of measurement
type, load type, kernel versions, configuration options, processors or SOCs.

I agree with you, we have some information on the wiki, but it seems
hard to find, seeing how people are asking again and again the same
questions about configuring an x86 kernel, which are answered and
hyperlinked in at least three pages on the web site.

So, if you feel like organizing things a bit better, I created a wiki
account for you.

-- 
                                                                Gilles.

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[hauptmech]

On 12/16/2012 03:11 PM, Gilles Chanteperdrix wrote:
> On 12/16/2012 02:41 PM, hauptmech wrote:
>
>> I'm coming to the documentation with relatively fresh eyes. If you give
>> me a user on the wiki I'm happy to do a little gardening. Things like
>> the [getting started] page which is useful but not linked to anymore.
>> Perhaps a buffer page between the wiki and the git page with a little
>> description of the repos and how they fit into the development workflow
>> (where a link to denx could go).
>
> The "getting started" page is unfinished (contains empty paragraphs),
> and outdated (contains dead links and references to things like the
> simulator which is currently not really usable) at the same time. I do
> not believe we have the resources to maintain such a page up-to-date.
Do you mind if your list of documentation wants goes into the wiki (at 
the taskmarket I guess)?

>
>  From my point of view, what we need is:
> - up to date manual pages of the binaries in xenomai distribution, in
> the xenomai sources, we can start from the current manual pages, but
> some of them are outdated and we should convert them to asciidoc so that
> more people can work on them, and we can put the html versions on the
> web site (with automatic update, as currently happens for the API
> documentation and README.INSTALL/TROUBLESHOOTING)
Do the asciidoc man page sources go in doc/asciidoc or doc/man?

I'm a bit shaky with automake/Makefile scripts, if someone can shim 
Makefile.am with the code to call 'a2x -f manpage $SOURCEFILE' in the 
right location then I don't mind converting them. An example...

CLOCKTEST(1)
============
:doctype: manpage

NAME
----
clocktest - Xenomai Clock Test

SYNOPSIS
--------
*clocktest* ['OPTIONS']

DESCRIPTION
-----------
*clocktest* is part of the Xenomai test suite and tests the Clock. For each
CPU, it repeatedly prints a time offset (compared to the reference
gettimeofday()), a drift value, the number of warps and the maximum warp in
microseconds.

For this program to work, you need to run a suitable Xenomai enabled kernel
with the respective module (xeno_posix).

OPTIONS
-------
*-C <clock_id>*::
     clock to be tested, default=0 (CLOCK_REALTIME=0, CLOCK_MONOTONIC=1, 
CLOCK_HOST_REALTIME=42)

*-T <test_duration_seconds>*::
     default=0 (Never stop, ^C to end)

*-D*::
     print extra diagnostics for CLOCK_HOST_REALTIME

AUTHOR
------

*clocktest* was written by Jan Kiszka <jan.kiszka@web.de>. This man page
was written by Roland Stigge <stigge@antcom.de>.

> - some thematic pages on the wiki such as:
> . calibrating your system with xeno-test
> . influence of the various Linux kernel options on latencies on at least
> the arm and x86 platforms
> - and hyperlinks, for instance at the end of README.INSTALL point to the
> "calibrating your system" page, and in this page, have an hyperlink to
> the xeno-test and latency test manual pages, in the page "porting the
> I-pipe-core patch to a new ARM board", mention the tools that can be
> used to debug the various issues, and have an hyperlink to their manual
> page.
> - a user guide, the "porting posix applications to xenomai" page, was my
> attempt of a user-guide, but it has a very specialized goal, whereas
> part of the information it contains is general to anyone writing posix
> applications for xenomai, and evern applications in general.
> - something we have talked about but has long gone forgotten, have some
> kind of database of the latency results sortable by a mix of measurement
> type, load type, kernel versions, configuration options, processors or SOCs.
>

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Gilles Chanteperdrix]

Hi,

On 12/17/2012 01:02 AM, hauptmech wrote:

> On 12/16/2012 03:11 PM, Gilles Chanteperdrix wrote:
>> On 12/16/2012 02:41 PM, hauptmech wrote:
>>
>>> I'm coming to the documentation with relatively fresh eyes. If you give
>>> me a user on the wiki I'm happy to do a little gardening. Things like
>>> the [getting started] page which is useful but not linked to anymore.
>>> Perhaps a buffer page between the wiki and the git page with a little
>>> description of the repos and how they fit into the development workflow
>>> (where a link to denx could go).
>>
>> The "getting started" page is unfinished (contains empty paragraphs),
>> and outdated (contains dead links and references to things like the
>> simulator which is currently not really usable) at the same time. I do
>> not believe we have the resources to maintain such a page up-to-date.
> Do you mind if your list of documentation wants goes into the wiki (at 
> the taskmarket I guess)?


Go ahead. Another task we talked about would be pages for users out of
tree ports, like currently exist for the Raspberry Pi, BeagleBone and
OMAP L138, where some information such as where to find the patch,
applying to which kernel sources and links to pre-built kernels if they
exist.

> 
>>
>>  From my point of view, what we need is:
>> - up to date manual pages of the binaries in xenomai distribution, in
>> the xenomai sources, we can start from the current manual pages, but
>> some of them are outdated and we should convert them to asciidoc so that
>> more people can work on them, and we can put the html versions on the
>> web site (with automatic update, as currently happens for the API
>> documentation and README.INSTALL/TROUBLESHOOTING)
> Do the asciidoc man page sources go in doc/asciidoc or doc/man?
> 
> I'm a bit shaky with automake/Makefile scripts, if someone can shim 
> Makefile.am with the code to call 'a2x -f manpage $SOURCEFILE' in the 
> right location then I don't mind converting them. An example...


Ok, I will do that.

> *clocktest* was written by Jan Kiszka <jan.kiszka@web.de>. This man page
> was written by Roland Stigge <stigge@antcom.de>.


Yes, that is fine, however we should remove the mail addresses (but keep
the authors addresses), are the pages are going to be on the website.

Thanks for helping with this.
Regards.

-- 
                                                                Gilles.

Re: [Xenomai] How does the adeos and xenomai project stay synchronized with the linux kernel project?

[Gilles Chanteperdrix]

On 12/17/2012 01:02 AM, hauptmech wrote:

> On 12/16/2012 03:11 PM, Gilles Chanteperdrix wrote:
>> On 12/16/2012 02:41 PM, hauptmech wrote:
>>
>>> I'm coming to the documentation with relatively fresh eyes. If you give
>>> me a user on the wiki I'm happy to do a little gardening. Things like
>>> the [getting started] page which is useful but not linked to anymore.
>>> Perhaps a buffer page between the wiki and the git page with a little
>>> description of the repos and how they fit into the development workflow
>>> (where a link to denx could go).
>>
>> The "getting started" page is unfinished (contains empty paragraphs),
>> and outdated (contains dead links and references to things like the
>> simulator which is currently not really usable) at the same time. I do
>> not believe we have the resources to maintain such a page up-to-date.
> Do you mind if your list of documentation wants goes into the wiki (at 
> the taskmarket I guess)?
> 
>>
>>  From my point of view, what we need is:
>> - up to date manual pages of the binaries in xenomai distribution, in
>> the xenomai sources, we can start from the current manual pages, but
>> some of them are outdated and we should convert them to asciidoc so that
>> more people can work on them, and we can put the html versions on the
>> web site (with automatic update, as currently happens for the API
>> documentation and README.INSTALL/TROUBLESHOOTING)
> Do the asciidoc man page sources go in doc/asciidoc or doc/man?
> 
> I'm a bit shaky with automake/Makefile scripts, if someone can shim 
> Makefile.am with the code to call 'a2x -f manpage $SOURCEFILE' in the 
> right location then I don't mind converting them. An example...


I have just pushed the changes to do that. I integrated the clocktest
example. To add the new foo.man manual page, you have to:
- remove the corresponding references to foo.man in configure.in
- add a foo.adoc file in the doc/man directory
- add foo to the HTML_DOCS variable (to generate the html version of the
manual page) in doc/man/Makefile.am
- add foo.adoc to the EXTRA_DIST variable in doc/man/Makefile.am
- run scripts/bootstrap

Then you should be able to run make in the build directory.

-- 
                                                                Gilles.