Null pointer dereference in tg3_poll_work running linux-3.4

Null pointer dereference in tg3_poll_work running linux-3.4

[Salam Noureddine]

Hi,

We've seen a very rare kernel panic in tg3_poll_work on hardware
running linux-3.4.
I haven't seen any upstream patches that seem to fix this issue in the
tg3 driver.
The disassembly shows that the panic is happening in tg3_rx which is
inlined into
tg3_poll_work. In the code below, the "data" pointer seem to be Null,

                        tg3_recycle_rx(tnapi, tpr, opaque_key,
                                       desc_idx, *post_ptr);

                        skb = netdev_alloc_skb(tp->dev,
                                               len + TG3_RAW_IP_ALIGN);

                        if (skb == NULL)
                                goto drop_it_no_recycle;

                        skb_reserve(skb, TG3_RAW_IP_ALIGN);
                        pci_dma_sync_single_for_cpu(tp->pdev,
dma_addr, len, PCI_DMA_FROMDEVICE);
                        memcpy(skb->data,
                               data + TG3_RX_OFFSET(tp),
                               len);

                        pci_dma_sync_single_for_device(tp->pdev, dma_addr, len,
PCI_DMA_FROMDEVICE);

I am wondering if anyone has seen this before or if it was fixed and I
missed the patch for it. If not,
any ideas on how we could end up with data being null? I don't have a
reproduction scenario for
this one.

Thanks,

Salam

Re: Null pointer dereference in tg3_poll_work running linux-3.4

[Michael Chan]

On Tue, Mar 28, 2017 at 11:32 AM, Salam Noureddine
<noureddine@arista.com> wrote:
> Hi,
>
> We've seen a very rare kernel panic in tg3_poll_work on hardware
> running linux-3.4.
> I haven't seen any upstream patches that seem to fix this issue in the
> tg3 driver.
> The disassembly shows that the panic is happening in tg3_rx which is
> inlined into
> tg3_poll_work. In the code below, the "data" pointer seem to be Null,

The data pointer is derived from the opaque value in the receive
descriptor.  When the hardware completes a receive packet, the opaque
value is returned so that the driver can retrieve the proper entry in
the ring and locate the "data" pointer for the buffer.

I don't remember any bug fixes related to this logic.  One possibility
is that there is DMA corruption and we are getting a bad opaque value.
If you have the core dump, it will be useful to look at the receive
completion ring.   These opaque values are simple index values and
should be in sequence.

Re: Null pointer dereference in tg3_poll_work running linux-3.4

[Salam Noureddine]

On Tue, Mar 28, 2017 at 11:53 AM, Michael Chan
<michael.chan@broadcom.com> wrote:

> I don't remember any bug fixes related to this logic.  One possibility
> is that there is DMA corruption and we are getting a bad opaque value.
> If you have the core dump, it will be useful to look at the receive
> completion ring.   These opaque values are simple index values and
> should be in sequence.

Unfortunately I don't have a core dump, this has happened around 5
times on boxes that have been up between 2 and 7 months. So it
seems extremely rare. So far, the only information I have is the
stack trace.