From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: Crash due to NULL dereference in tcp_rearm_rto Date: Fri, 13 Apr 2018 18:14:09 -0700 Message-ID: <86c0f836-9a7c-438b-d81a-839be45f1f58@gmail.com> References: <0e7dbdef0ecf0ee0562a144050165453@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit To: Subash Abhinov Kasiviswanathan , netdev@vger.kernel.org, soheil@google.com, eric.dumazet@gmail.com Return-path: Received: from mail-pl0-f68.google.com ([209.85.160.68]:33710 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750867AbeDNBOL (ORCPT ); Fri, 13 Apr 2018 21:14:11 -0400 Received: by mail-pl0-f68.google.com with SMTP id w12-v6so6076180plp.0 for ; Fri, 13 Apr 2018 18:14:10 -0700 (PDT) In-Reply-To: <0e7dbdef0ecf0ee0562a144050165453@codeaurora.org> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 04/13/2018 05:39 PM, Subash Abhinov Kasiviswanathan wrote: > We are seeing a warning followed by a crash on an ARM64 device with > Android 4.14 based kernel. > > It looks like both sk->sk_write_queue and sk->sk_send_head are NULL. > Since the sk->sk_write_queue is NULL and is dereferenced in tcp_rto_delta_us() > to get the skb->skb_mstamp, there is crash observed. > > Since this is 4.14.32, it already has commit ("tcp: reset sk_send_head in tcp_write_queue_purge") > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.14.34&id=dbbf2d1e4077bab0c65ece2765d3fc69cf7d610f > > 12876.013077:   <6> WARNING: CPU: 5 PID: 14828 at net/ipv4/tcp_output.c:2469 tcp_send_loss_probe+0x198/0x1b8 > 12876.038939:   <6> task: ffffffe73f7e5a80 task.stack: ffffff801b068000 > 12876.038941:   <2> PC is at tcp_send_loss_probe+0x198/0x1b8 > 12876.038942:   <2> LR is at tcp_send_loss_probe+0x28/0x1b8 > 12876.038944:   <2> pc : [] lr : [] pstate: 60400145 > 12876.038944:   <2> sp : ffffff800802bd30 > 12876.038945:   <2> x29: ffffff800802bd50 x28: ffffff8dc9d83eb8 > 12876.038948:   <2> x27: ffffff800802be08 x26: ffffff8dc9737000 > 12876.038950:   <2> x25: 0000000000000001 x24: ffffffe744ea1728 > 12876.038952:   <2> x23: ffffffe73f7e5a80 x22: 0000000000000558 > 12876.038954:   <2> x21: 0000000000000000 x20: 0000000001080020 > 12876.038956:   <2> x19: ffffffe73d06e440 x18: 0000000000000020 > 12876.038958:   <2> x17: 0000000000000014 x16: 0000000000000030 > 12876.038960:   <2> x15: 0000000000000000 x14: 0000000000000000 > 12876.038962:   <2> x13: 0000000013af314c x12: 0000002773f8a550 > 12876.038965:   <2> x11: 0000000000000538 x10: 0000000000000000 > 12876.038967:   <2> x9 : 0000000000000020 x8 : ffffffe73d06e5f0 > 12876.038969:   <2> x7 : 0000000000924278 x6 : ffffffe76fe9ed80 > 12876.038971:   <2> x5 : ffffffe76fe9ed80 x4 : 000000000f500458 > 12876.038973:   <2> x3 : ffffff800802bce0 x2 : ffffff800802bce8 > 12876.038975:   <2> x1 : 0000000000000000 x0 : 0000000000000558 > 12876.039082:   <2> [] tcp_send_loss_probe+0x198/0x1b8 > 12876.039084:   <2> [] tcp_write_timer_handler+0xf8/0x1c4 > 12876.039086:   <2> [] tcp_write_timer+0x5c/0x98 > 12876.039089:   <2> [] call_timer_fn+0xc0/0x1b4 > 12876.039091:   <2> [] run_timer_softirq+0x230/0x850 > 12876.039094:   <2> [] __do_softirq+0x1dc/0x3a4 > 12876.039096:   <2> [] irq_exit+0xc8/0xd4 > 12876.039098:   <2> [] __handle_domain_irq+0x8c/0xc4 > 12876.039099:   <2> [] gic_handle_irq+0x164/0x1bc > > [net/ipv4/tcp_output.c] > void tcp_send_loss_probe(struct sock *sk) > { >     struct tcp_sock *tp = tcp_sk(sk); >     struct sk_buff *skb; >     int pcount; >     int mss = tcp_current_mss(sk); > > ... > >     /* Retransmit last segment. */ >     if (WARN_ON(!skb)) >         goto rearm_timer; > > 12876.043967:   <6> Unable to handle kernel NULL pointer dereference at virtual address 00000010 > 12876.091600:   <6> Internal error: Oops: 96000005 [#1] PREEMPT SMP > 12876.152597:   <2> PC is at tcp_rearm_rto+0x48/0x90 > 12876.156979:   <2> LR is at tcp_send_loss_probe+0x178/0x1b8 > 12876.162077:   <2> pc : [] lr : [] pstate: 60400145 > 12876.169657:   <2> sp : ffffff800802bd10 > 12876.173056:   <2> x29: ffffff800802bd20 x28: ffffff8dc9d83eb8 > 12876.178511:   <2> x27: ffffff800802be08 x26: ffffff8dc9737000 > 12876.183967:   <2> x25: 0000000000000001 x24: ffffffe744ea1728 > 12876.189418:   <2> x23: ffffffe73f7e5a80 x22: 0000000000000558 > 12876.194863:   <2> x21: 0000000000000000 x20: 0000000001080020 > 12876.200312:   <2> x19: ffffffe73d06e440 x18: 0000000000000020 > 12876.205758:   <2> x17: 0000000000000014 x16: 0000000000000030 > 12876.211212:   <2> x15: 0000000000000000 x14: 0000000000000000 > 12876.216660:   <2> x13: 0000000013af314c x12: 0000002773f8a550 > 12876.222108:   <2> x11: 0000000000000538 x10: 0000000000000000 > 12876.227561:   <2> x9 : 0000000000000000 x8 : ffffffe73d06e5f0 > 12876.233008:   <2> x7 : 0000000000924278 x6 : ffffffe76fe9ed80 > 12876.238455:   <2> x5 : ffffffe76fe9ed80 x4 : 000000000f500458 > 12876.243907:   <2> x3 : ffffff800802bce0 x2 : ffffff800802bce8 > 12876.249360:   <2> x1 : 0000000000000000 x0 : 0000000000000867 > 12876.473522:   <2> [] tcp_rearm_rto+0x48/0x90 > 12876.478971:   <2> [] tcp_send_loss_probe+0x178/0x1b8 > 12876.485131:   <2> [] tcp_write_timer_handler+0xf8/0x1c4 > 12876.491557:   <2> [] tcp_write_timer+0x5c/0x98 > 12876.497189:   <2> [] call_timer_fn+0xc0/0x1b4 > 12876.502731:   <2> [] run_timer_softirq+0x230/0x850 > 12876.508716:   <2> [] __do_softirq+0x1dc/0x3a4 > 12876.514260:   <2> [] irq_exit+0xc8/0xd4 > 12876.519261:   <2> [] __handle_domain_irq+0x8c/0xc4 > 12876.525245:   <2> [] gic_handle_irq+0x164/0x1bc > > [net/ipv4/tcp_input.c] > void tcp_rearm_rto(struct sock *sk) > { > ... >         inet_csk_clear_xmit_timer(sk, ICSK_TIME_RETRANS); >     } else { >         u32 rto = inet_csk(sk)->icsk_rto; >         /* Offset the time elapsed after installing regular RTO */ >         if (icsk->icsk_pending == ICSK_TIME_REO_TIMEOUT || >             icsk->icsk_pending == ICSK_TIME_LOSS_PROBE) { >             s64 delta_us = tcp_rto_delta_us(sk); >             /* delta_us may not be positive if the socket is locked >              * when the retrans timer fires and is rescheduled. >              */ >             rto = usecs_to_jiffies(max_t(int, delta_us, 1)); > > > [include/net/tcp.h] > static inline s64 tcp_rto_delta_us(const struct sock *sk) > { >     const struct sk_buff *skb = tcp_write_queue_head(sk); >     u32 rto = inet_csk(sk)->icsk_rto; >     u64 rto_time_stamp_us = skb->skb_mstamp + jiffies_to_usecs(rto); > >     return rto_time_stamp_us - tcp_sk(sk)->tcp_mstamp; > } > We probably need to clear tp->packets_out :/