* Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself
@ 2018-03-14 15:02 Stephen Hemminger
2018-03-15 17:51 ` Guillaume Nault
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2018-03-14 15:02 UTC (permalink / raw)
To: xeb; +Cc: netdev
Begin forwarded message:
Date: Wed, 14 Mar 2018 06:56:09 +0000
From: bugzilla-daemon@bugzilla.kernel.org
To: stephen@networkplumber.org
Subject: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself
https://bugzilla.kernel.org/show_bug.cgi?id=199109
Bug ID: 199109
Summary: pptp: kernel printk "recursion detected", and then
reboot itself
Product: Networking
Version: 2.5
Kernel Version: 4.9.77
Hardware: Mips32
OS: Linux
Tree: Mainline
Status: NEW
Severity: high
Priority: P1
Component: IPV4
Assignee: stephen@networkplumber.org
Reporter: xuheng333@zoho.com
Regression: No
Created attachment 274715
--> https://bugzilla.kernel.org/attachment.cgi?id=274715&action=edit
system log
Use openwrt LEDE, (gcc version 5.5.0 (OpenWrt GCC 5.5.0 r5932-9c4fe10) ).
When use pptp, make WAN(eth1) down and then make WAN(eth1) up, pptp will redial
itself. Do this in a loop.
After a while, kernel print "recursion detected" and "net_ratelimit: 49422
callbacks suppressed", print this many times, then system reboot.
When add more printk in driver "driver/net/ppp/ppp_generic.c", fond it fall in
loop in "static void __ppp_xmit_process(struct ppp *ppp)", which called by
"static void __ppp_channel_push(struct channel *pch)".
__ppp_channel_push() locked ppp->xmit_recursion, and loop for long time( while
(!ppp->xmit_pending && (skb = skb_dequeue(&ppp->file.xq))) ppp_send_frame(ppp,
skb); ). Some other thread want to lock ppp->xmit_recursion, but failed.
static void __ppp_channel_push(struct channel *pch)
{
struct sk_buff *skb;
struct ppp *ppp;
spin_lock_bh(&pch->downl);
if (pch->chan) {
while (!skb_queue_empty(&pch->file.xq)) {
skb = skb_dequeue(&pch->file.xq);
if (!pch->chan->ops->start_xmit(pch->chan, skb)) {
/* put the packet back and try again later */
skb_queue_head(&pch->file.xq, skb);
break;
}
}
} else {
/* channel got deregistered */
skb_queue_purge(&pch->file.xq);
}
spin_unlock_bh(&pch->downl);
/* see if there is anything from the attached unit to be sent */
if (skb_queue_empty(&pch->file.xq)) {
ppp = pch->ppp;
if (ppp)
__ppp_xmit_process(ppp);
}
}
--
You are receiving this mail because:
You are the assignee for the bug.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself 2018-03-14 15:02 Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself Stephen Hemminger @ 2018-03-15 17:51 ` Guillaume Nault [not found] ` <1622d9239ec.fb8c9df225155.9220077804249612652@zoho.com> 0 siblings, 1 reply; 6+ messages in thread From: Guillaume Nault @ 2018-03-15 17:51 UTC (permalink / raw) To: xuheng333, Stephen Hemminger; +Cc: xeb, netdev On Wed, Mar 14, 2018 at 08:02:17AM -0700, Stephen Hemminger wrote: > > > Begin forwarded message: > > Date: Wed, 14 Mar 2018 06:56:09 +0000 > From: bugzilla-daemon@bugzilla.kernel.org > To: stephen@networkplumber.org > Subject: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself > > > https://bugzilla.kernel.org/show_bug.cgi?id=199109 > > Bug ID: 199109 > Summary: pptp: kernel printk "recursion detected", and then > reboot itself > Product: Networking > Version: 2.5 > Kernel Version: 4.9.77 > Hardware: Mips32 > OS: Linux > Tree: Mainline > Status: NEW > Severity: high > Priority: P1 > Component: IPV4 > Assignee: stephen@networkplumber.org > Reporter: xuheng333@zoho.com > Regression: No > > Created attachment 274715 > --> https://bugzilla.kernel.org/attachment.cgi?id=274715&action=edit > system log > > Use openwrt LEDE, (gcc version 5.5.0 (OpenWrt GCC 5.5.0 r5932-9c4fe10) ). > When use pptp, make WAN(eth1) down and then make WAN(eth1) up, pptp will redial > itself. Do this in a loop. > > After a while, kernel print "recursion detected" and "net_ratelimit: 49422 > callbacks suppressed", print this many times, then system reboot. > > When add more printk in driver "driver/net/ppp/ppp_generic.c", fond it fall in > loop in "static void __ppp_xmit_process(struct ppp *ppp)", which called by > "static void __ppp_channel_push(struct channel *pch)". > __ppp_channel_push() locked ppp->xmit_recursion, and loop for long time( while > (!ppp->xmit_pending && (skb = skb_dequeue(&ppp->file.xq))) ppp_send_frame(ppp, > skb); ). Some other thread want to lock ppp->xmit_recursion, but failed. > > Hi Xuheng, Are you sure that your PPTP packets aren't routed back to your PPP interface? What's the IP address of the PPTP server? Regards, Guillaume ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <1622d9239ec.fb8c9df225155.9220077804249612652@zoho.com>]
* Re: Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself [not found] ` <1622d9239ec.fb8c9df225155.9220077804249612652@zoho.com> @ 2018-03-16 20:02 ` Guillaume Nault 2018-03-20 15:38 ` Guillaume Nault 0 siblings, 1 reply; 6+ messages in thread From: Guillaume Nault @ 2018-03-16 20:02 UTC (permalink / raw) To: xu heng; +Cc: Stephen Hemminger, xeb, netdev On Fri, Mar 16, 2018 at 02:49:40PM +0800, xu heng wrote: > > For testing, in __ppp_channel_push(), disable sending anything from the attached unit, just disable __ppp_xmit_process(ppp) in __ppp_channel_push(). In my opinion, __ppp_xmit_process() should only called by ppp_xmit_process(), because of ppp->xmit_recursion __percpu. > ppp_channel_push() needs to call __ppp_xmit_process() because some drivers (like ppp_async) need to notify ppp_generic when they can accept new packets. This is done by ppp_output_wakeup() which then calls ppp_channel_push(). So we have to handle the unit backlog there. Please try the following patch (untested). diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c index b883af93929c..af22eb11bbaa 100644 --- a/drivers/net/ppp/ppp_generic.c +++ b/drivers/net/ppp/ppp_generic.c @@ -255,7 +255,7 @@ struct ppp_net { /* Prototypes. */ static int ppp_unattached_ioctl(struct net *net, struct ppp_file *pf, struct file *file, unsigned int cmd, unsigned long arg); -static void ppp_xmit_process(struct ppp *ppp); +static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb); static void ppp_send_frame(struct ppp *ppp, struct sk_buff *skb); static void ppp_push(struct ppp *ppp); static void ppp_channel_push(struct channel *pch); @@ -511,13 +511,12 @@ static ssize_t ppp_write(struct file *file, const char __user *buf, goto out; } - skb_queue_tail(&pf->xq, skb); - switch (pf->kind) { case INTERFACE: - ppp_xmit_process(PF_TO_PPP(pf)); + ppp_xmit_process(PF_TO_PPP(pf), skb); break; case CHANNEL: + skb_queue_tail(&pf->xq, skb); ppp_channel_push(PF_TO_CHANNEL(pf)); break; } @@ -1260,8 +1259,8 @@ ppp_start_xmit(struct sk_buff *skb, struct net_device *dev) put_unaligned_be16(proto, pp); skb_scrub_packet(skb, !net_eq(ppp->ppp_net, dev_net(dev))); - skb_queue_tail(&ppp->file.xq, skb); - ppp_xmit_process(ppp); + ppp_xmit_process(ppp, skb); + return NETDEV_TX_OK; outf: @@ -1415,13 +1414,14 @@ static void ppp_setup(struct net_device *dev) */ /* Called to do any work queued up on the transmit side that can now be done */ -static void __ppp_xmit_process(struct ppp *ppp) +static void __ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb) { - struct sk_buff *skb; - ppp_xmit_lock(ppp); if (!ppp->closing) { ppp_push(ppp); + + if (skb) + skb_queue_tail(&ppp->file.xq, skb); while (!ppp->xmit_pending && (skb = skb_dequeue(&ppp->file.xq))) ppp_send_frame(ppp, skb); @@ -1435,7 +1435,7 @@ static void __ppp_xmit_process(struct ppp *ppp) ppp_xmit_unlock(ppp); } -static void ppp_xmit_process(struct ppp *ppp) +static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb) { local_bh_disable(); @@ -1443,7 +1443,7 @@ static void ppp_xmit_process(struct ppp *ppp) goto err; (*this_cpu_ptr(ppp->xmit_recursion))++; - __ppp_xmit_process(ppp); + __ppp_xmit_process(ppp, skb); (*this_cpu_ptr(ppp->xmit_recursion))--; local_bh_enable(); @@ -1452,6 +1452,7 @@ static void ppp_xmit_process(struct ppp *ppp) err: local_bh_enable(); + kfree_skb(skb); if (net_ratelimit()) netdev_err(ppp->dev, "recursion detected\n"); @@ -1937,7 +1938,7 @@ static void __ppp_channel_push(struct channel *pch) if (skb_queue_empty(&pch->file.xq)) { ppp = pch->ppp; if (ppp) - __ppp_xmit_process(ppp); + __ppp_xmit_process(ppp, NULL); } } ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself 2018-03-16 20:02 ` Guillaume Nault @ 2018-03-20 15:38 ` Guillaume Nault [not found] ` <1624615812e.11e307c748584.6445977831797927207@zoho.com> 0 siblings, 1 reply; 6+ messages in thread From: Guillaume Nault @ 2018-03-20 15:38 UTC (permalink / raw) To: xu heng; +Cc: Stephen Hemminger, xeb, netdev On Fri, Mar 16, 2018 at 09:02:40PM +0100, Guillaume Nault wrote: > On Fri, Mar 16, 2018 at 02:49:40PM +0800, xu heng wrote: > > > > For testing, in __ppp_channel_push(), disable sending anything from the attached unit, just disable __ppp_xmit_process(ppp) in __ppp_channel_push(). In my opinion, __ppp_xmit_process() should only called by ppp_xmit_process(), because of ppp->xmit_recursion __percpu. > > > ppp_channel_push() needs to call __ppp_xmit_process() because some > drivers (like ppp_async) need to notify ppp_generic when they can > accept new packets. This is done by ppp_output_wakeup() which then > calls ppp_channel_push(). So we have to handle the unit backlog there. > > > Please try the following patch (untested). > FYI, I've now fully tested the patch. I'm going to send it formally. > diff --git a/drivers/net/ppp/ppp_generic.c b/drivers/net/ppp/ppp_generic.c > index b883af93929c..af22eb11bbaa 100644 > --- a/drivers/net/ppp/ppp_generic.c > +++ b/drivers/net/ppp/ppp_generic.c > @@ -255,7 +255,7 @@ struct ppp_net { > /* Prototypes. */ > static int ppp_unattached_ioctl(struct net *net, struct ppp_file *pf, > struct file *file, unsigned int cmd, unsigned long arg); > -static void ppp_xmit_process(struct ppp *ppp); > +static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb); > static void ppp_send_frame(struct ppp *ppp, struct sk_buff *skb); > static void ppp_push(struct ppp *ppp); > static void ppp_channel_push(struct channel *pch); > @@ -511,13 +511,12 @@ static ssize_t ppp_write(struct file *file, const char __user *buf, > goto out; > } > > - skb_queue_tail(&pf->xq, skb); > - > switch (pf->kind) { > case INTERFACE: > - ppp_xmit_process(PF_TO_PPP(pf)); > + ppp_xmit_process(PF_TO_PPP(pf), skb); > break; > case CHANNEL: > + skb_queue_tail(&pf->xq, skb); > ppp_channel_push(PF_TO_CHANNEL(pf)); > break; > } > @@ -1260,8 +1259,8 @@ ppp_start_xmit(struct sk_buff *skb, struct net_device *dev) > put_unaligned_be16(proto, pp); > > skb_scrub_packet(skb, !net_eq(ppp->ppp_net, dev_net(dev))); > - skb_queue_tail(&ppp->file.xq, skb); > - ppp_xmit_process(ppp); > + ppp_xmit_process(ppp, skb); > + > return NETDEV_TX_OK; > > outf: > @@ -1415,13 +1414,14 @@ static void ppp_setup(struct net_device *dev) > */ > > /* Called to do any work queued up on the transmit side that can now be done */ > -static void __ppp_xmit_process(struct ppp *ppp) > +static void __ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb) > { > - struct sk_buff *skb; > - > ppp_xmit_lock(ppp); > if (!ppp->closing) { > ppp_push(ppp); > + > + if (skb) > + skb_queue_tail(&ppp->file.xq, skb); > while (!ppp->xmit_pending && > (skb = skb_dequeue(&ppp->file.xq))) > ppp_send_frame(ppp, skb); > @@ -1435,7 +1435,7 @@ static void __ppp_xmit_process(struct ppp *ppp) > ppp_xmit_unlock(ppp); > } > > -static void ppp_xmit_process(struct ppp *ppp) > +static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb) > { > local_bh_disable(); > > @@ -1443,7 +1443,7 @@ static void ppp_xmit_process(struct ppp *ppp) > goto err; > > (*this_cpu_ptr(ppp->xmit_recursion))++; > - __ppp_xmit_process(ppp); > + __ppp_xmit_process(ppp, skb); > (*this_cpu_ptr(ppp->xmit_recursion))--; > > local_bh_enable(); > @@ -1452,6 +1452,7 @@ static void ppp_xmit_process(struct ppp *ppp) > > err: > local_bh_enable(); > + kfree_skb(skb); > > if (net_ratelimit()) > netdev_err(ppp->dev, "recursion detected\n"); > @@ -1937,7 +1938,7 @@ static void __ppp_channel_push(struct channel *pch) > if (skb_queue_empty(&pch->file.xq)) { > ppp = pch->ppp; > if (ppp) > - __ppp_xmit_process(ppp); > + __ppp_xmit_process(ppp, NULL); > } > } > > ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <1624615812e.11e307c748584.6445977831797927207@zoho.com>]
* Re: Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself [not found] ` <1624615812e.11e307c748584.6445977831797927207@zoho.com> @ 2018-03-21 8:35 ` Guillaume Nault 2018-03-22 2:41 ` xu heng 0 siblings, 1 reply; 6+ messages in thread From: Guillaume Nault @ 2018-03-21 8:35 UTC (permalink / raw) To: xu heng; +Cc: Stephen Hemminger, xeb, netdev On Wed, Mar 21, 2018 at 09:03:57AM +0800, xu heng wrote: > Yes, i have tested it for 146390 seconds in my board, it's ok now. Thanks! > Feel free to add your Tested-by tag to the patch if you want to. Thanks for your report. Guillaume BTW, for your future exchanges on the list, please avoid top-posting. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself 2018-03-21 8:35 ` Guillaume Nault @ 2018-03-22 2:41 ` xu heng 0 siblings, 0 replies; 6+ messages in thread From: xu heng @ 2018-03-22 2:41 UTC (permalink / raw) To: Guillaume Nault; +Cc: Stephen Hemminger, xeb, netdev ---- On Wed, 21 Mar 2018 16:35:28 +0800 Guillaume Nault <g.nault@alphalink.fr> wrote ---- > On Wed, Mar 21, 2018 at 09:03:57AM +0800, xu heng wrote: > > Yes, i have tested it for 146390 seconds in my board, it's ok now. Thanks! > > > Feel free to add your Tested-by tag to the patch if you want to. > Thanks for your report. > > Guillaume > > BTW, for your future exchanges on the list, please avoid top-posting. > I'm sorry for that, will never do that again. Thanks. xuheng ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-03-22 2:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-03-14 15:02 Fw: [Bug 199109] New: pptp: kernel printk "recursion detected", and then reboot itself Stephen Hemminger
2018-03-15 17:51 ` Guillaume Nault
[not found] ` <1622d9239ec.fb8c9df225155.9220077804249612652@zoho.com>
2018-03-16 20:02 ` Guillaume Nault
2018-03-20 15:38 ` Guillaume Nault
[not found] ` <1624615812e.11e307c748584.6445977831797927207@zoho.com>
2018-03-21 8:35 ` Guillaume Nault
2018-03-22 2:41 ` xu heng
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).