* Re: [PATCH 5.10 104/157] mptcp: put subflow sock on connect error [not found] ` <20210322121937.071435221@linuxfoundation.org> @ 2021-03-24 8:32 ` Naresh Kamboju 2021-03-24 9:04 ` Florian Westphal 2021-03-24 9:26 ` Greg Kroah-Hartman 0 siblings, 2 replies; 5+ messages in thread From: Naresh Kamboju @ 2021-03-24 8:32 UTC (permalink / raw) To: Greg Kroah-Hartman, Florian Westphal, David S. Miller Cc: open list, linux-stable, Mat Martineau, Sasha Levin, Anders Roxell, lkft-triage, Netdev, Arnd Bergmann On Mon, 22 Mar 2021 at 18:15, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote: > > From: Florian Westphal <fw@strlen.de> > > [ Upstream commit f07157792c633b528de5fc1dbe2e4ea54f8e09d4 ] > > mptcp_add_pending_subflow() performs a sock_hold() on the subflow, > then adds the subflow to the join list. > > Without a sock_put the subflow sk won't be freed in case connect() fails. > > unreferenced object 0xffff88810c03b100 (size 3000): > [..] > sk_prot_alloc.isra.0+0x2f/0x110 > sk_alloc+0x5d/0xc20 > inet6_create+0x2b7/0xd30 > __sock_create+0x17f/0x410 > mptcp_subflow_create_socket+0xff/0x9c0 > __mptcp_subflow_connect+0x1da/0xaf0 > mptcp_pm_nl_work+0x6e0/0x1120 > mptcp_worker+0x508/0x9a0 > > Fixes: 5b950ff4331ddda ("mptcp: link MPC subflow into msk only after accept") > Signed-off-by: Florian Westphal <fw@strlen.de> > Signed-off-by: Mat Martineau <mathew.j.martineau@linux.intel.com> > Signed-off-by: David S. Miller <davem@davemloft.net> > Signed-off-by: Sasha Levin <sashal@kernel.org> I have reported the following warnings and kernel crash on 5.10.26-rc2 [1] The bisect reported that issue pointing out to this commit. commit 460916534896e6d4f80a37152e0948db33376873 mptcp: put subflow sock on connect error This problem is specific to 5.10.26-rc2. Warning: -------- [ 1040.114695] refcount_t: addition on 0; use-after-free. [ 1040.119857] WARNING: CPU: 3 PID: 31925 at /usr/src/kernel/lib/refcount.c:25 refcount_warn_saturate+0xd7/0x100 [ 1040.129769] Modules linked in: act_mirred cls_u32 sch_netem sch_etf ip6table_nat xt_nat iptable_nat nf_nat ip6table_filter xt_conntrack nf_conntrack nf_defrag_ipv4 libcrc32c ip6_tables nf_defrag_ipv6 sch_fq iptable_filter xt_mark ip_tables cls_bpf sch_ingress algif_hash x86_pkg_temp_thermal fuse [last unloaded: test_blackhole_dev] [ 1040.159030] CPU: 3 PID: 31925 Comm: mptcp_connect Tainted: G W K 5.10.26-rc2 #1 [ 1040.167459] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS 2.2 05/23/2018 [ 1040.174851] RIP: 0010:refcount_warn_saturate+0xd7/0x100 And Kernel Panic: ------------- [ 1069.557485] BUG: kernel NULL pointer dereference, address: 0000000000000010 [ 1069.564446] #PF: supervisor read access in kernel mode [ 1069.569583] #PF: error_code(0x0000) - not-present page [ 1069.574714] PGD 0 P4D 0 [ 1069.577246] Oops: 0000 [#1] SMP PTI [ 1069.580730] CPU: 1 PID: 17 Comm: ksoftirqd/1 Tainted: G W K 5.10.26-rc2 #1 [ 1069.588719] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS 2.2 05/23/2018 [ 1069.596106] RIP: 0010:selinux_socket_sock_rcv_skb+0x3f/0x290 ... [ 1069.961697] Kernel panic - not syncing: Fatal exception in interrupt [ 1069.968083] Kernel Offset: 0x18600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) steps to reproduce: -------------------------- - cd /opt/kselftests/mainline/net/mptcp - ./mptcp_join.sh || true Reported-by: Naresh Kamboju <naresh.kamboju@linaro.org> crash test link: https://lkft.validation.linaro.org/scheduler/job/2436164 Revert this commit and test job: https://lkft.validation.linaro.org/scheduler/job/2437401#L1207 > --- > net/mptcp/subflow.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c > index 16adba172fb9..591546d0953f 100644 > --- a/net/mptcp/subflow.c > +++ b/net/mptcp/subflow.c > @@ -1133,6 +1133,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc, > spin_lock_bh(&msk->join_list_lock); > list_add_tail(&subflow->node, &msk->join_list); > spin_unlock_bh(&msk->join_list_lock); > + sock_put(mptcp_subflow_tcp_sock(subflow)); > > return err; > url: [1] https://lore.kernel.org/stable/20210323182123.3ce89282@yaviniv.e18.physik.tu-muenchen.de/T/#m7994b86b52391a746e7d5be214885a5a1b2f9713 -- Linaro LKFT https://lkft.linaro.org ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 5.10 104/157] mptcp: put subflow sock on connect error 2021-03-24 8:32 ` [PATCH 5.10 104/157] mptcp: put subflow sock on connect error Naresh Kamboju @ 2021-03-24 9:04 ` Florian Westphal 2021-03-24 9:22 ` Greg Kroah-Hartman 2021-03-24 9:26 ` Greg Kroah-Hartman 1 sibling, 1 reply; 5+ messages in thread From: Florian Westphal @ 2021-03-24 9:04 UTC (permalink / raw) To: Naresh Kamboju Cc: Greg Kroah-Hartman, Florian Westphal, David S. Miller, open list, linux-stable, Mat Martineau, Sasha Levin, Anders Roxell, lkft-triage, Netdev, Arnd Bergmann Naresh Kamboju <naresh.kamboju@linaro.org> wrote: > On Mon, 22 Mar 2021 at 18:15, Greg Kroah-Hartman > <gregkh@linuxfoundation.org> wrote: > > > > From: Florian Westphal <fw@strlen.de> > > > > [ Upstream commit f07157792c633b528de5fc1dbe2e4ea54f8e09d4 ] > > > > mptcp_add_pending_subflow() performs a sock_hold() on the subflow, > > then adds the subflow to the join list. > > > > Without a sock_put the subflow sk won't be freed in case connect() fails. > > > > unreferenced object 0xffff88810c03b100 (size 3000): > > [..] > > sk_prot_alloc.isra.0+0x2f/0x110 > > sk_alloc+0x5d/0xc20 > > inet6_create+0x2b7/0xd30 > > __sock_create+0x17f/0x410 > > mptcp_subflow_create_socket+0xff/0x9c0 > > __mptcp_subflow_connect+0x1da/0xaf0 > > mptcp_pm_nl_work+0x6e0/0x1120 > > mptcp_worker+0x508/0x9a0 > > > > Fixes: 5b950ff4331ddda ("mptcp: link MPC subflow into msk only after accept") I don't see this change in 5.10, so why is this fix queued up? > I have reported the following warnings and kernel crash on 5.10.26-rc2 [1] > The bisect reported that issue pointing out to this commit. > > commit 460916534896e6d4f80a37152e0948db33376873 > mptcp: put subflow sock on connect error > > This problem is specific to 5.10.26-rc2. > > Warning: > -------- > [ 1040.114695] refcount_t: addition on 0; use-after-free. > [ 1040.119857] WARNING: CPU: 3 PID: 31925 at > /usr/src/kernel/lib/refcount.c:25 refcount_warn_saturate+0xd7/0x100 > [ 1040.129769] Modules linked in: act_mirred cls_u32 sch_netem sch_etf > ip6table_nat xt_nat iptable_nat nf_nat ip6table_filter xt_conntrack > nf_conntrack nf_defrag_ipv4 libcrc32c ip6_tables nf_defrag_ipv6 sch_fq > iptable_filter xt_mark ip_tables cls_bpf sch_ingress algif_hash > x86_pkg_temp_thermal fuse [last unloaded: test_blackhole_dev] > [ 1040.159030] CPU: 3 PID: 31925 Comm: mptcp_connect Tainted: G > W K 5.10.26-rc2 #1 > [ 1040.167459] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS > 2.2 05/23/2018 > [ 1040.174851] RIP: 0010:refcount_warn_saturate+0xd7/0x100 > > And > > Kernel Panic: > ------------- > [ 1069.557485] BUG: kernel NULL pointer dereference, address: 0000000000000010 > [ 1069.564446] #PF: supervisor read access in kernel mode > [ 1069.569583] #PF: error_code(0x0000) - not-present page > [ 1069.574714] PGD 0 P4D 0 > [ 1069.577246] Oops: 0000 [#1] SMP PTI > > index 16adba172fb9..591546d0953f 100644 > > --- a/net/mptcp/subflow.c > > +++ b/net/mptcp/subflow.c > > @@ -1133,6 +1133,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc, > > spin_lock_bh(&msk->join_list_lock); > > list_add_tail(&subflow->node, &msk->join_list); > > spin_unlock_bh(&msk->join_list_lock); > > + sock_put(mptcp_subflow_tcp_sock(subflow)); > > > > return err; Crash is not surprising, the backport puts the socket in the 'success' path (list_add_tail). I don't see why this is in -stable, the faulty commit is not there? The upstream patch is: list_del(&subflow->node); spin_unlock_bh(&msk->join_list_lock); + sock_put(mptcp_subflow_tcp_sock(subflow)); [ Note the 'list_del', this is in the error unwind path ] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 5.10 104/157] mptcp: put subflow sock on connect error 2021-03-24 9:04 ` Florian Westphal @ 2021-03-24 9:22 ` Greg Kroah-Hartman 2021-03-24 14:28 ` Sasha Levin 0 siblings, 1 reply; 5+ messages in thread From: Greg Kroah-Hartman @ 2021-03-24 9:22 UTC (permalink / raw) To: Florian Westphal Cc: Naresh Kamboju, David S. Miller, open list, linux-stable, Mat Martineau, Sasha Levin, Anders Roxell, lkft-triage, Netdev, Arnd Bergmann On Wed, Mar 24, 2021 at 10:04:12AM +0100, Florian Westphal wrote: > Naresh Kamboju <naresh.kamboju@linaro.org> wrote: > > On Mon, 22 Mar 2021 at 18:15, Greg Kroah-Hartman > > <gregkh@linuxfoundation.org> wrote: > > > > > > From: Florian Westphal <fw@strlen.de> > > > > > > [ Upstream commit f07157792c633b528de5fc1dbe2e4ea54f8e09d4 ] > > > > > > mptcp_add_pending_subflow() performs a sock_hold() on the subflow, > > > then adds the subflow to the join list. > > > > > > Without a sock_put the subflow sk won't be freed in case connect() fails. > > > > > > unreferenced object 0xffff88810c03b100 (size 3000): > > > [..] > > > sk_prot_alloc.isra.0+0x2f/0x110 > > > sk_alloc+0x5d/0xc20 > > > inet6_create+0x2b7/0xd30 > > > __sock_create+0x17f/0x410 > > > mptcp_subflow_create_socket+0xff/0x9c0 > > > __mptcp_subflow_connect+0x1da/0xaf0 > > > mptcp_pm_nl_work+0x6e0/0x1120 > > > mptcp_worker+0x508/0x9a0 > > > > > > Fixes: 5b950ff4331ddda ("mptcp: link MPC subflow into msk only after accept") > > I don't see this change in 5.10, so why is this fix queued up? > > > I have reported the following warnings and kernel crash on 5.10.26-rc2 [1] > > The bisect reported that issue pointing out to this commit. > > > > commit 460916534896e6d4f80a37152e0948db33376873 > > mptcp: put subflow sock on connect error > > > > This problem is specific to 5.10.26-rc2. > > > > Warning: > > -------- > > [ 1040.114695] refcount_t: addition on 0; use-after-free. > > [ 1040.119857] WARNING: CPU: 3 PID: 31925 at > > /usr/src/kernel/lib/refcount.c:25 refcount_warn_saturate+0xd7/0x100 > > [ 1040.129769] Modules linked in: act_mirred cls_u32 sch_netem sch_etf > > ip6table_nat xt_nat iptable_nat nf_nat ip6table_filter xt_conntrack > > nf_conntrack nf_defrag_ipv4 libcrc32c ip6_tables nf_defrag_ipv6 sch_fq > > iptable_filter xt_mark ip_tables cls_bpf sch_ingress algif_hash > > x86_pkg_temp_thermal fuse [last unloaded: test_blackhole_dev] > > [ 1040.159030] CPU: 3 PID: 31925 Comm: mptcp_connect Tainted: G > > W K 5.10.26-rc2 #1 > > [ 1040.167459] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS > > 2.2 05/23/2018 > > [ 1040.174851] RIP: 0010:refcount_warn_saturate+0xd7/0x100 > > > > And > > > > Kernel Panic: > > ------------- > > [ 1069.557485] BUG: kernel NULL pointer dereference, address: 0000000000000010 > > [ 1069.564446] #PF: supervisor read access in kernel mode > > [ 1069.569583] #PF: error_code(0x0000) - not-present page > > [ 1069.574714] PGD 0 P4D 0 > > [ 1069.577246] Oops: 0000 [#1] SMP PTI > > > index 16adba172fb9..591546d0953f 100644 > > > --- a/net/mptcp/subflow.c > > > +++ b/net/mptcp/subflow.c > > > @@ -1133,6 +1133,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc, > > > spin_lock_bh(&msk->join_list_lock); > > > list_add_tail(&subflow->node, &msk->join_list); > > > spin_unlock_bh(&msk->join_list_lock); > > > + sock_put(mptcp_subflow_tcp_sock(subflow)); > > > > > > return err; > > Crash is not surprising, the backport puts the socket in the 'success' path > (list_add_tail). > > I don't see why this is in -stable, the faulty commit is not there? > > The upstream patch is: > list_del(&subflow->node); > spin_unlock_bh(&msk->join_list_lock); > + sock_put(mptcp_subflow_tcp_sock(subflow)); > > [ Note the 'list_del', this is in the error unwind path ] Odd, I think something went wrong with Sasha's scripts. I've dropped this, and the other two mptcp patches, from the 5.10 queue and let's see if that helps. I'll do a new -rc now as well after my build tests finish... thanks, greg k-h ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 5.10 104/157] mptcp: put subflow sock on connect error 2021-03-24 9:22 ` Greg Kroah-Hartman @ 2021-03-24 14:28 ` Sasha Levin 0 siblings, 0 replies; 5+ messages in thread From: Sasha Levin @ 2021-03-24 14:28 UTC (permalink / raw) To: Greg Kroah-Hartman Cc: Florian Westphal, Naresh Kamboju, David S. Miller, open list, linux-stable, Mat Martineau, Anders Roxell, lkft-triage, Netdev, Arnd Bergmann On Wed, Mar 24, 2021 at 10:22:29AM +0100, Greg Kroah-Hartman wrote: >On Wed, Mar 24, 2021 at 10:04:12AM +0100, Florian Westphal wrote: >> Naresh Kamboju <naresh.kamboju@linaro.org> wrote: >> > On Mon, 22 Mar 2021 at 18:15, Greg Kroah-Hartman >> > <gregkh@linuxfoundation.org> wrote: >> > > >> > > From: Florian Westphal <fw@strlen.de> >> > > >> > > [ Upstream commit f07157792c633b528de5fc1dbe2e4ea54f8e09d4 ] >> > > >> > > mptcp_add_pending_subflow() performs a sock_hold() on the subflow, >> > > then adds the subflow to the join list. >> > > >> > > Without a sock_put the subflow sk won't be freed in case connect() fails. >> > > >> > > unreferenced object 0xffff88810c03b100 (size 3000): >> > > [..] >> > > sk_prot_alloc.isra.0+0x2f/0x110 >> > > sk_alloc+0x5d/0xc20 >> > > inet6_create+0x2b7/0xd30 >> > > __sock_create+0x17f/0x410 >> > > mptcp_subflow_create_socket+0xff/0x9c0 >> > > __mptcp_subflow_connect+0x1da/0xaf0 >> > > mptcp_pm_nl_work+0x6e0/0x1120 >> > > mptcp_worker+0x508/0x9a0 >> > > >> > > Fixes: 5b950ff4331ddda ("mptcp: link MPC subflow into msk only after accept") >> >> I don't see this change in 5.10, so why is this fix queued up? >> >> > I have reported the following warnings and kernel crash on 5.10.26-rc2 [1] >> > The bisect reported that issue pointing out to this commit. >> > >> > commit 460916534896e6d4f80a37152e0948db33376873 >> > mptcp: put subflow sock on connect error >> > >> > This problem is specific to 5.10.26-rc2. >> > >> > Warning: >> > -------- >> > [ 1040.114695] refcount_t: addition on 0; use-after-free. >> > [ 1040.119857] WARNING: CPU: 3 PID: 31925 at >> > /usr/src/kernel/lib/refcount.c:25 refcount_warn_saturate+0xd7/0x100 >> > [ 1040.129769] Modules linked in: act_mirred cls_u32 sch_netem sch_etf >> > ip6table_nat xt_nat iptable_nat nf_nat ip6table_filter xt_conntrack >> > nf_conntrack nf_defrag_ipv4 libcrc32c ip6_tables nf_defrag_ipv6 sch_fq >> > iptable_filter xt_mark ip_tables cls_bpf sch_ingress algif_hash >> > x86_pkg_temp_thermal fuse [last unloaded: test_blackhole_dev] >> > [ 1040.159030] CPU: 3 PID: 31925 Comm: mptcp_connect Tainted: G >> > W K 5.10.26-rc2 #1 >> > [ 1040.167459] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS >> > 2.2 05/23/2018 >> > [ 1040.174851] RIP: 0010:refcount_warn_saturate+0xd7/0x100 >> > >> > And >> > >> > Kernel Panic: >> > ------------- >> > [ 1069.557485] BUG: kernel NULL pointer dereference, address: 0000000000000010 >> > [ 1069.564446] #PF: supervisor read access in kernel mode >> > [ 1069.569583] #PF: error_code(0x0000) - not-present page >> > [ 1069.574714] PGD 0 P4D 0 >> > [ 1069.577246] Oops: 0000 [#1] SMP PTI >> > > index 16adba172fb9..591546d0953f 100644 >> > > --- a/net/mptcp/subflow.c >> > > +++ b/net/mptcp/subflow.c >> > > @@ -1133,6 +1133,7 @@ int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_addr_info *loc, >> > > spin_lock_bh(&msk->join_list_lock); >> > > list_add_tail(&subflow->node, &msk->join_list); >> > > spin_unlock_bh(&msk->join_list_lock); >> > > + sock_put(mptcp_subflow_tcp_sock(subflow)); >> > > >> > > return err; >> >> Crash is not surprising, the backport puts the socket in the 'success' path >> (list_add_tail). >> >> I don't see why this is in -stable, the faulty commit is not there? >> >> The upstream patch is: >> list_del(&subflow->node); >> spin_unlock_bh(&msk->join_list_lock); >> + sock_put(mptcp_subflow_tcp_sock(subflow)); >> >> [ Note the 'list_del', this is in the error unwind path ] > >Odd, I think something went wrong with Sasha's scripts. I brought in the commit it depends on as a dependency for something else, but ended up throwing it away, forgetting about this commit. Sorry :( -- Thanks, Sasha ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 5.10 104/157] mptcp: put subflow sock on connect error 2021-03-24 8:32 ` [PATCH 5.10 104/157] mptcp: put subflow sock on connect error Naresh Kamboju 2021-03-24 9:04 ` Florian Westphal @ 2021-03-24 9:26 ` Greg Kroah-Hartman 1 sibling, 0 replies; 5+ messages in thread From: Greg Kroah-Hartman @ 2021-03-24 9:26 UTC (permalink / raw) To: Naresh Kamboju Cc: Florian Westphal, David S. Miller, open list, linux-stable, Mat Martineau, Sasha Levin, Anders Roxell, lkft-triage, Netdev, Arnd Bergmann On Wed, Mar 24, 2021 at 02:02:06PM +0530, Naresh Kamboju wrote: > On Mon, 22 Mar 2021 at 18:15, Greg Kroah-Hartman > <gregkh@linuxfoundation.org> wrote: > > > > From: Florian Westphal <fw@strlen.de> > > > > [ Upstream commit f07157792c633b528de5fc1dbe2e4ea54f8e09d4 ] > > > > mptcp_add_pending_subflow() performs a sock_hold() on the subflow, > > then adds the subflow to the join list. > > > > Without a sock_put the subflow sk won't be freed in case connect() fails. > > > > unreferenced object 0xffff88810c03b100 (size 3000): > > [..] > > sk_prot_alloc.isra.0+0x2f/0x110 > > sk_alloc+0x5d/0xc20 > > inet6_create+0x2b7/0xd30 > > __sock_create+0x17f/0x410 > > mptcp_subflow_create_socket+0xff/0x9c0 > > __mptcp_subflow_connect+0x1da/0xaf0 > > mptcp_pm_nl_work+0x6e0/0x1120 > > mptcp_worker+0x508/0x9a0 > > > > Fixes: 5b950ff4331ddda ("mptcp: link MPC subflow into msk only after accept") > > Signed-off-by: Florian Westphal <fw@strlen.de> > > Signed-off-by: Mat Martineau <mathew.j.martineau@linux.intel.com> > > Signed-off-by: David S. Miller <davem@davemloft.net> > > Signed-off-by: Sasha Levin <sashal@kernel.org> > > I have reported the following warnings and kernel crash on 5.10.26-rc2 [1] > The bisect reported that issue pointing out to this commit. > > commit 460916534896e6d4f80a37152e0948db33376873 > mptcp: put subflow sock on connect error > > This problem is specific to 5.10.26-rc2. Thank you for tracking this down! ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-03-24 14:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20210322121933.746237845@linuxfoundation.org>
[not found] ` <20210322121937.071435221@linuxfoundation.org>
2021-03-24 8:32 ` [PATCH 5.10 104/157] mptcp: put subflow sock on connect error Naresh Kamboju
2021-03-24 9:04 ` Florian Westphal
2021-03-24 9:22 ` Greg Kroah-Hartman
2021-03-24 14:28 ` Sasha Levin
2021-03-24 9:26 ` Greg Kroah-Hartman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).