From: Marc Kleine-Budde <mkl@pengutronix.de>
To: "Glen, Andrew" <AGlen@bepmarine.com>
Cc: "linux-can@vger.kernel.org" <linux-can@vger.kernel.org>
Subject: Re: Calls to 'pm_runtime_resume()' made from the interrupt context without 'pm_runtime_irq_safe' in 'c_can.c' causing occasional kernel crash.
Date: Sun, 24 Nov 2013 23:51:46 +0100 [thread overview]
Message-ID: <52928302.3010007@pengutronix.de> (raw)
In-Reply-To: <2A43324E02E34840B044C9DC922144800904D233@AMDTCEX10.actuant.pri>
[-- Attachment #1: Type: text/plain, Size: 4353 bytes --]
On 11/20/2013 03:07 AM, Glen, Andrew wrote:
> I've been getting occasional crashes running the 3.12 mainline kernel
> with a Debian rootfs on an AM335x (Beaglebone Black).
> The crash would only ever occur the first time the can0 device is
> brought up, i.e.: Reboot, call to 'ip link set can0 type can bitrate
> 250000 triple-sampling on', then 'ip link set can0 up' and about 40%
> of the time I'd get a crash.
> If the 'ip link set can0 up' succeeded I could then bring the can0
> device up/down repeatedly with no issues.
> Strangely, this problem would not occur running the same kernel with
> the Ubuntu rootfs - but I think this is unrelated to the issue.
> As the crash dump suggests, the problem is caused by a call to
> '__pm_runtime_resume' being made from within an interrupt routine -
> occuring when the CAN device is brought up immediately into a bus
> error (assumedly caused by starting up the can receiver midway
> through a transmission by another device).
Can you provide the crash dump?
> Power management should not occur from within an interrupt routine,
> so by making a special interrupt version of the routine
> problematically calling '__pm_runtime_resume' (i.e.
> 'c_can_get_berr_counter'), but with this call removed, I was
> initially able to get around the problem.
> Further researching into the issue, I found the following in
> '/Documentation/power/runtime_pm.txt':
> It is safe to execute the following helper functions from interrupt context:
>
> pm_request_idle()
> pm_request_autosuspend()
> pm_schedule_suspend()
> pm_request_resume()
> pm_runtime_get_noresume()
> pm_runtime_get()
> pm_runtime_put_noidle()
> pm_runtime_put()
> pm_runtime_put_autosuspend()
> pm_runtime_enable()
> pm_suspend_ignore_children()
> pm_runtime_set_active()
> pm_runtime_set_suspended()
> pm_runtime_suspended()
> pm_runtime_mark_last_busy()
> pm_runtime_autosuspend_expiration()
>
> If pm_runtime_irq_safe() has been called for a device then the
> following helper functions may also be used in interrupt context:
>
> pm_runtime_idle()
> pm_runtime_suspend()
> pm_runtime_autosuspend()
> pm_runtime_resume()
> pm_runtime_get_sync()
> pm_runtime_put_sync()
> pm_runtime_put_sync_suspend()
> pm_runtime_put_sync_autosuspend()
>
>
> In 'c_can.c' as we find it in 3.12 mainline, calls to
> 'pm_runtime_resume()' are being made from the interrupt context (i.e.
> omap3_intc_handle_irq
> ->handle_IRQ->irq_exit->do_softirq->__do_softirq->net_rx_action->c_can_poll->c_can_state_change->c_can_get_berr_counter->__pm_runtime_resume)
> without 'pm_runtime_irq_safe' having first been called.
> So it makes sense that these problems were occurring, and that the
> problem goes away after inserting this call.
>
> I propose something like the following to fix this problem:
>
> diff --git a/drivers/net/can/c_can/c_can.c b/drivers/net/can/c_can/c_can.c index a668cd4..638fd22 100644
> --- a/drivers/net/can/c_can/c_can.c
> +++ b/drivers/net/can/c_can/c_can.c
> <at> <at> -1103,6 +1103,8 <at> <at> static int c_can_open(struct net_device *dev)
> int err;
> struct c_can_priv *priv = netdev_priv(dev);
>
> + pm_runtime_irq_safe(priv->device);
> +
> c_can_pm_runtime_get_sync(priv);
> c_can_reset_ram(priv, true);
Thanks for the detailed analysis. The problem with the
c_can_get_berr_counter() function is, that is is called from IRQ handler
and from the userspace via the priv->can.do_get_berr_counter callback.
If the c_can_get_berr_counter() is called from the IRQ handler, there is
no need to call c_can_pm_runtime_get_sync(), as the device is up the
open() function has been called. As the callback can be called if the
device is down, we need the c_can_pm_runtime_get_sync() call.
From my point of view (I'm not familiar with runtime pm)
pm_runtime_irq_safe() has some drawbacks. I think we have a better
option, by introducing __c_can_get_berr_counter() which will not call
into the runtime PM.
Marc
--
Pengutronix e.K. | Marc Kleine-Budde |
Industrial Linux Solutions | Phone: +49-231-2826-924 |
Vertretung West/Dortmund | Fax: +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686 | http://www.pengutronix.de |
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]
next prev parent reply other threads:[~2013-11-24 22:51 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-11-20 2:07 Calls to 'pm_runtime_resume()' made from the interrupt context without 'pm_runtime_irq_safe' in 'c_can.c' causing occasional kernel crash Glen, Andrew
2013-11-24 22:51 ` Marc Kleine-Budde [this message]
2013-11-25 0:32 ` Glen, Andrew
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52928302.3010007@pengutronix.de \
--to=mkl@pengutronix.de \
--cc=AGlen@bepmarine.com \
--cc=linux-can@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox