[Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Terje Frøysa]

Dear Forum,

Occasionally I get the warning listed below on my Beaglebone Black console.
I am running Debian 3.8.13-bone67 with Xenomai 2.6.4

I am running four RTDM modules handling interrupts from GPIO pins.

As far as I have found, this kind of warnings can be associated with inappropriate calls from user space.
But I cannot find such calls in my code.
The irq is requested by a RTDM kernel module when issuing a device open call from user space.

Please advice
Best regards
Terje Froysa


[ 398.167710] RTDM BARO INT (GPIO 60) - open
[  398.167730] RTDM BARO pin 60 - irq number 188
[  398.167738] RTDM BARO 60 - request GPIO
[  398.167757] RTDM BARO 60 - request IRQ 188
[  398.167768] ------------[ cut here ]------------
[  398.172673] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()
[  398.180951] Modules linked in: g_multi libcomposite imu_rtdm(O) baro_rtdm(O) gyro_rtdm(O) rtgps_1pps(O) omap_rng
[  398.191733] [<c00117ed>] (unwind_backtrace+0x1/0x9c) from [<c003022b>] (warn_slowpath_common+0x33/0x48)
[  398.201606] [<c003022b>] (warn_slowpath_common+0x33/0x48) from [<c0030251>] (warn_slowpath_null+0x11/0x14)
[  398.211701] [<c0030251>] (warn_slowpath_null+0x11/0x14) from [<c001470f>] (ipipe_set_irq_affinity+0x47/0x70)
[  398.222010] [<c001470f>] (ipipe_set_irq_affinity+0x47/0x70) from [<c00991b5>] (xnintr_attach+0x21/0x1f0)
[  398.231936] [<c00991b5>] (xnintr_attach+0x21/0x1f0) from [<c00eea8d>] (rtdm_irq_request+0x25/0x44)
[  398.241324] [<c00eea8d>] (rtdm_irq_request+0x25/0x44) from [<bf80c153>] (baro_irq_open+0x6a/0xfc [baro_rtdm])
[  398.251704] [<bf80c153>] (baro_irq_open+0x6a/0xfc [baro_rtdm]) from [<c00eb531>] (__rt_dev_open+0x61/0xa4)
[  398.261832] [<c00eb531>] (__rt_dev_open+0x61/0xa4) from [<c00eeca9>] (sys_rtdm_open+0x3d/0x48)
[  398.270853] [<c00eeca9>] (sys_rtdm_open+0x3d/0x48) from [<c00a9649>] (losyscall_event+0x81/0x1e0)
[  398.280142] [<c00a9649>] (losyscall_event+0x81/0x1e0) from [<c007d169>] (ipipe_syscall_hook+0x2d/0x34)
[  398.289892] [<c007d169>] (ipipe_syscall_hook+0x2d/0x34) from [<c007bfc7>] (__ipipe_notify_syscall+0x5b/0xb8)
[  398.300180] [<c007bfc7>] (__ipipe_notify_syscall+0x5b/0xb8) from [<c001487d>] (__ipipe_syscall_root+0x3d/0x98)
[  398.310646] [<c001487d>] (__ipipe_syscall_root+0x3d/0x98) from [<c000cbcd>] (local_restart+0x9/0x54)
[  398.320194] ---[ end trace f174ab8361e31708 ]---

Smail: Terje Frøysa, SINTEF ICT (Information and Communication Technology), N-7465 Trondheim, Norway.
Email: terje.froysa@sintef.no<blocked::mailto:terje.froysa@sintef.no> <mailto:terje.froysa@sintef.no<blocked::mailto:terje.froysa@sintef.no>>
Phone: +47 930 03603,
Company: Independent R&D, see <http://www.sintef.no/<blocked::http://www.sintef.no/>

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 05:54:36AM +0000, Terje Frøysa wrote:
> Dear Forum,
> 
> Occasionally I get the warning listed below on my Beaglebone Black
> console. I am running Debian 3.8.13-bone67 with Xenomai 2.6.4
> 
> I am running four RTDM modules handling interrupts from GPIO pins.
> 
> As far as I have found, this kind of warnings can be associated
> with inappropriate calls from user space. But I cannot find such
> calls in my code. The irq is requested by a RTDM kernel module
> when issuing a device open call from user space.

Yes, there are several such issues in Xenomai 2.x. Note however that
the Beaglebone is uniprocessor, whereas ipip_set_irq_affinity is
called when CONFIG_SMP was enabled. Enabling CONFIG_SMP for an
uniprocessor is not recommended with Xenomai 2.x, as it gives
slightly worst performances.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Terje Frøysa]

Thanks Gilles,

I don't know why the debian kernel for the Sitara have SMP enabled.
It sounds like a good idea to turn it off then...

Best regards
Terje

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: 1. desember 2014 08:35
To: Terje Frøysa
Cc: xenomai@xenomai.org
Subject: Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

On Mon, Dec 01, 2014 at 05:54:36AM +0000, Terje Frøysa wrote:
> Dear Forum,
> 
> Occasionally I get the warning listed below on my Beaglebone Black 
> console. I am running Debian 3.8.13-bone67 with Xenomai 2.6.4
> 
> I am running four RTDM modules handling interrupts from GPIO pins.
> 
> As far as I have found, this kind of warnings can be associated with 
> inappropriate calls from user space. But I cannot find such calls in 
> my code. The irq is requested by a RTDM kernel module when issuing a 
> device open call from user space.

Yes, there are several such issues in Xenomai 2.x. Note however that the Beaglebone is uniprocessor, whereas ipip_set_irq_affinity is called when CONFIG_SMP was enabled. Enabling CONFIG_SMP for an uniprocessor is not recommended with Xenomai 2.x, as it gives slightly worst performances.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 08:26:37AM +0000, Terje Frøysa wrote:
> Thanks Gilles,
> 
> I don't know why the debian kernel for the Sitara have SMP enabled.
> It sounds like a good idea to turn it off then...

Note that if you want to be able to recompile your kernel fast, you
should not be using a Debian kernel configuration anyway, but a
configuration with just the options you need enabled. This makes a
big difference on compilation time, a small difference on memory
footprint, and a big difference on root file system footprint.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Terje Frøysa]

Thanks Gilles,

The Debian/Xenomai was solely chosen as a start-up scenario since I found a good recipe on the net.
I am heading for Yocto and LFS (Linux From Scratch) for future embedded systems.

By the way.. When compiling kernel for Xenomai, I am advised to turn off CPU Frequency scaling "as it is known to cause trouble".
When enabled I could inspect and set the CPU clock frequency by the "cpufreq-info" and "cpufreq-set".
These tools are no longer available and I don't find any settings in the menuconfig.

What is the default BBB CPU frequency when frequency scaling is turned off?

Best regards
Terje

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: 1. desember 2014 09:50
To: Terje Frøysa
Cc: xenomai@xenomai.org
Subject: Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

On Mon, Dec 01, 2014 at 08:26:37AM +0000, Terje Frøysa wrote:
> Thanks Gilles,
> 
> I don't know why the debian kernel for the Sitara have SMP enabled.
> It sounds like a good idea to turn it off then...

Note that if you want to be able to recompile your kernel fast, you should not be using a Debian kernel configuration anyway, but a configuration with just the options you need enabled. This makes a big difference on compilation time, a small difference on memory footprint, and a big difference on root file system footprint.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 11:06:58AM +0000, Terje Frøysa wrote:
> Thanks Gilles,
> 
> The Debian/Xenomai was solely chosen as a start-up scenario since I found a good recipe on the net.
> I am heading for Yocto and LFS (Linux From Scratch) for future embedded systems.
> 
> By the way.. When compiling kernel for Xenomai, I am advised to
> turn off CPU Frequency scaling "as it is known to cause trouble".

Yes, for two reasons: 

- because Xenomai timing system is counting on a fixed tsc (and
hardware timer) frequency. In the case of beaglebone, the tsc is
based on the gptimer, so if the gptimer frequency does not change
when changing the processor OPP, this is probably not an issue 

- because the execution time of real-time tasks change, this remains
an issue.


> When enabled I could inspect and set the CPU clock frequency by
> the "cpufreq-info" and "cpufreq-set". These tools are no longer
> available and I don't find any settings in the menuconfig.

These are part of the cpufreq-utils package.

> 
> What is the default BBB CPU frequency when frequency scaling is turned off?

All bets are off. This depends on the kernel implementation. On
OMAP3 (which is closed to BBB, normally), last time I checked (and
that was a long time ago), the kernel does not touch whatever PLL
settings U-boot did, with regard to cpu frequency.

If the kernel provides an access to the PLL settings in /proc, you
can check the various PLL frequencies and look-up in the processor
reference manual to what OPP this corresponds.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Hidde Verstoep]

> By the way.. When compiling kernel for Xenomai, I am advised to turn off CPU Frequency scaling "as it is known to cause trouble".
> When enabled I could inspect and set the CPU clock frequency by the "cpufreq-info" and "cpufreq-set".
> These tools are no longer available and I don't find any settings in the menuconfig.
>
> What is the default BBB CPU frequency when frequency scaling is turned off?

>From my experience with the BBB and Xenomai:
The default CPU frequency with frequency scaling turned off is 600
MHz. This was not acceptable for me since the BBB supports 1000 Mhz.
So, I left frequency scaling on in the kernel, but set the default
mode to "performance" (always runs the CPU at max frequency). Issue
with this is that the mode still gets reset to "ondemand" every now
and then (I have no clue as to why). So, I configured the kernel with
frequency scaling on, the default mode set to "performance" and all
other modes disabled. This seems to work without issues with Xenomai.

Kind regards,
Hidde Verstoep

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Terje Frøysa]

Thanks for the tip Hidde!

Best regards
Terje Froysa

-----Original Message-----
From: Hidde Verstoep [mailto:hverstoep@e2mtechnologies.eu] 
Sent: 1. desember 2014 12:35
To: Terje Frøysa
Cc: Gilles Chanteperdrix; xenomai@xenomai.org
Subject: Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

> By the way.. When compiling kernel for Xenomai, I am advised to turn off CPU Frequency scaling "as it is known to cause trouble".
> When enabled I could inspect and set the CPU clock frequency by the "cpufreq-info" and "cpufreq-set".
> These tools are no longer available and I don't find any settings in the menuconfig.
>
> What is the default BBB CPU frequency when frequency scaling is turned off?

From my experience with the BBB and Xenomai:
The default CPU frequency with frequency scaling turned off is 600 MHz. This was not acceptable for me since the BBB supports 1000 Mhz.
So, I left frequency scaling on in the kernel, but set the default mode to "performance" (always runs the CPU at max frequency). Issue with this is that the mode still gets reset to "ondemand" every now and then (I have no clue as to why). So, I configured the kernel with frequency scaling on, the default mode set to "performance" and all other modes disabled. This seems to work without issues with Xenomai.

Kind regards,
Hidde Verstoep

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Lennart Sorensen]

On Mon, Dec 01, 2014 at 05:54:36AM +0000, Terje Frøysa wrote:
> Occasionally I get the warning listed below on my Beaglebone Black console.
> I am running Debian 3.8.13-bone67 with Xenomai 2.6.4
> 
> I am running four RTDM modules handling interrupts from GPIO pins.
> 
> As far as I have found, this kind of warnings can be associated with inappropriate calls from user space.
> But I cannot find such calls in my code.
> The irq is requested by a RTDM kernel module when issuing a device open call from user space.

I just changed the code to not give the warning.  The omap-gpio driver
doesn't support setting affinity, and the linux irq driver has no issue
with that, so perhaps xenomai/ipipe has no reason to have a problem with
it either.

-- 
Len Sorensen

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 10:19:51AM -0500, Lennart Sorensen wrote:
> On Mon, Dec 01, 2014 at 05:54:36AM +0000, Terje Frøysa wrote:
> > Occasionally I get the warning listed below on my Beaglebone Black console.
> > I am running Debian 3.8.13-bone67 with Xenomai 2.6.4
> > 
> > I am running four RTDM modules handling interrupts from GPIO pins.
> > 
> > As far as I have found, this kind of warnings can be associated with inappropriate calls from user space.
> > But I cannot find such calls in my code.
> > The irq is requested by a RTDM kernel module when issuing a device open call from user space.
> 
> I just changed the code to not give the warning.  The omap-gpio driver
> doesn't support setting affinity, and the linux irq driver has no issue
> with that, so perhaps xenomai/ipipe has no reason to have a problem with
> it either.

On the other hand, this is just a warning, it does not prevent the
system from working, and this warning warns us about a real problem:
we can not set the irq affinity of this driver.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Terje Frøysa]

Ok Len,

Thanks for your tip.
The warning comes every now and then, but I have never experienced any consequences from it.
So you might be quite right.

Terje

-----Original Message-----
From: Lennart Sorensen [mailto:lsorense@csclub.uwaterloo.ca] 
Sent: 1. desember 2014 16:20
To: Terje Frøysa
Cc: xenomai@xenomai.org
Subject: Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

On Mon, Dec 01, 2014 at 05:54:36AM +0000, Terje Frøysa wrote:
> Occasionally I get the warning listed below on my Beaglebone Black console.
> I am running Debian 3.8.13-bone67 with Xenomai 2.6.4
> 
> I am running four RTDM modules handling interrupts from GPIO pins.
> 
> As far as I have found, this kind of warnings can be associated with inappropriate calls from user space.
> But I cannot find such calls in my code.
> The irq is requested by a RTDM kernel module when issuing a device open call from user space.

I just changed the code to not give the warning.  The omap-gpio driver doesn't support setting affinity, and the linux irq driver has no issue with that, so perhaps xenomai/ipipe has no reason to have a problem with it either.

--
Len Sorensen

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 12:35:18PM +0100, Hidde Verstoep wrote:
> > By the way.. When compiling kernel for Xenomai, I am advised to
> > turn off CPU Frequency scaling "as it is known to cause
> > trouble". When enabled I could inspect and set the CPU clock
> > frequency by the "cpufreq-info" and "cpufreq-set". These tools
> > are no longer available and I don't find any settings in the
> > menuconfig.
> >
> > What is the default BBB CPU frequency when frequency scaling is turned off?
> 
> From my experience with the BBB and Xenomai:
> The default CPU frequency with frequency scaling turned off is 600
> MHz. This was not acceptable for me since the BBB supports 1000
> Mhz.

Well, to what frequency does u-boot set the processor? Because if
u-boot sets the frequency to 600 MHz and Linux does not change it,
chances are that if you set the frequency to 1 GHz in u-boot, Linux
will not change it either. This is what I do on the omap3 board I
use: set the frequency to 720 MHz in u-boot.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Lennart Sorensen]

On Mon, Dec 01, 2014 at 04:30:57PM +0100, Gilles Chanteperdrix wrote:
> On the other hand, this is just a warning, it does not prevent the
> system from working, and this warning warns us about a real problem:
> we can not set the irq affinity of this driver.

It was ugly and just a warning so I commented it out.

I don't think the hardware has any way to do affinity, so it can't be
fixed, so the warning doesn't help.

-- 
Len Sorensen

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 06:35:54PM -0500, Lennart Sorensen wrote:
> On Mon, Dec 01, 2014 at 04:30:57PM +0100, Gilles Chanteperdrix wrote:
> > On the other hand, this is just a warning, it does not prevent the
> > system from working, and this warning warns us about a real problem:
> > we can not set the irq affinity of this driver.
> 
> It was ugly and just a warning so I commented it out.
> 
> I don't think the hardware has any way to do affinity, so it can't be
> fixed, so the warning doesn't help.

Yes, but now you know it, without the message, you would not know it.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Lennart Sorensen]

On Tue, Dec 02, 2014 at 12:37:20AM +0100, Gilles Chanteperdrix wrote:
> On Mon, Dec 01, 2014 at 06:35:54PM -0500, Lennart Sorensen wrote:
> > On Mon, Dec 01, 2014 at 04:30:57PM +0100, Gilles Chanteperdrix wrote:
> > > On the other hand, this is just a warning, it does not prevent the
> > > system from working, and this warning warns us about a real problem:
> > > we can not set the irq affinity of this driver.
> > 
> > It was ugly and just a warning so I commented it out.
> > 
> > I don't think the hardware has any way to do affinity, so it can't be
> > fixed, so the warning doesn't help.
> 
> Yes, but now you know it, without the message, you would not know it.

True, but it sure looks terrible in the logs given it looks about the
same as a kernel crash.

And I still don't think there is anything that can be done about it.
You can set the affinity of the GIC irq for the whole GPIO bank, but
not for the individual pins of the gpio bank.

-- 
Len Sorensen

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Gilles Chanteperdrix]

On Mon, Dec 01, 2014 at 06:50:43PM -0500, Lennart Sorensen wrote:
> On Tue, Dec 02, 2014 at 12:37:20AM +0100, Gilles Chanteperdrix wrote:
> > On Mon, Dec 01, 2014 at 06:35:54PM -0500, Lennart Sorensen wrote:
> > > On Mon, Dec 01, 2014 at 04:30:57PM +0100, Gilles Chanteperdrix wrote:
> > > > On the other hand, this is just a warning, it does not prevent the
> > > > system from working, and this warning warns us about a real problem:
> > > > we can not set the irq affinity of this driver.
> > > 
> > > It was ugly and just a warning so I commented it out.
> > > 
> > > I don't think the hardware has any way to do affinity, so it can't be
> > > fixed, so the warning doesn't help.
> > 
> > Yes, but now you know it, without the message, you would not know it.
> 
> True, but it sure looks terrible in the logs given it looks about the
> same as a kernel crash.

To the uninformed yes. Kernel bugs begin with "BUG", kernel faults
(which would be what is nearest from a kernel crash) begin with
"Unhandled fault in kernel mode". Not with "Warning:".

> 
> And I still don't think there is anything that can be done about it.
> You can set the affinity of the GIC irq for the whole GPIO bank, but
> not for the individual pins of the gpio bank.

What I mean is that if you are writing a driver and you care about
irq latency, this information is important, it tells you can not pin
the irq handler to a particular core, and so that if you plan to
have a good latency, you should probably find an irq source which is
directly tied to the GIC, and avoid the GPIOs.

-- 
					    Gilles.

Re: [Xenomai] WARNING: at arch/arm/kernel/ipipe.c:157 ipipe_set_irq_affinity+0x47/0x70()

[Lennart Sorensen]

On Tue, Dec 02, 2014 at 12:55:36AM +0100, Gilles Chanteperdrix wrote:
> What I mean is that if you are writing a driver and you care about
> irq latency, this information is important, it tells you can not pin
> the irq handler to a particular core, and so that if you plan to
> have a good latency, you should probably find an irq source which is
> directly tied to the GIC, and avoid the GPIOs.

That is certainly true.  The gpio irqs on the omap chips really seem
most suited to things like push buttons and cable connected detection
and such where speed isn't an issue.

-- 
Len Sorensen