From: "Keller, Jacob E" <jacob.e.keller@intel.com>
To: "gpiccoli@linux.vnet.ibm.com" <gpiccoli@linux.vnet.ibm.com>,
"Kirsher, Jeffrey T" <jeffrey.t.kirsher@intel.com>,
"intel-wired-lan@lists.osuosl.org"
<intel-wired-lan@lists.osuosl.org>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [Intel-wired-lan] [PATCH net] i40e: avoid NULL pointer dereference and recursive errors on early PCI error
Date: Tue, 27 Sep 2016 22:41:59 +0000 [thread overview]
Message-ID: <1475016118.22933.1.camel@intel.com> (raw)
In-Reply-To: <1475010871-31682-1-git-send-email-gpiccoli@linux.vnet.ibm.com>
On Tue, 2016-09-27 at 18:14 -0300, Guilherme G. Piccoli wrote:
> Although rare, it's possible to hit PCI error early on device
> probe, meaning possibly some structs are not entirely initialized,
> and some might even be completely uninitialized, leading to NULL
> pointer dereference.
>
> The i40e driver currently presents a "bad" behavior if device hits
> such early PCI error: firstly, the struct i40e_pf might not be
> attached to pci_dev yet, leading to a NULL pointer dereference on
> access to pf->state.
>
Oops! Nice find!
> Even checking if the struct is NULL and avoiding the access in that
> case isn't enough, since the driver cannot recover from PCI error
> that early; in our experiments we saw multiple failures on kernel
> log, like:
>
> [549.664] i40e 0007:01:00.1: Initial pf_reset failed: -15
> [549.664] i40e: probe of 0007:01:00.1 failed with error -15
> [...]
> [871.644] i40e 0007:01:00.1: The driver for the device stopped
> because the
> device firmware failed to init. Try updating your NVM image.
> [871.644] i40e: probe of 0007:01:00.1 failed with error -32
> [...]
> [872.516] i40e 0007:01:00.0: ARQ: Unknown event 0x0000 ignored
>
> Between the first probe failure (error -15) and the second (error
> -32)
> another PCI error happened due to the first bad probe. Also, driver
> started to flood console with those ARQ event messages.
>
> This patch will prevent these issues by allowing error recovery
> mechanism to remove the failed device from the system instead of
> trying to recover from early PCI errors during device probe.
>
This seems reasonable.
> Signed-off-by: Guilherme G. Piccoli <gpiccoli@linux.vnet.ibm.com>
> ---
> drivers/net/ethernet/intel/i40e/i40e_main.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c
> b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index d0b3a1b..dad15b6 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -11360,6 +11360,12 @@ static pci_ers_result_t
> i40e_pci_error_detected(struct pci_dev *pdev,
>
> dev_info(&pdev->dev, "%s: error %d\n", __func__, error);
>
> + if (!pf) {
> + dev_info(&pdev->dev,
> + "Cannot recover - error happened during
> device probe\n");
> + return PCI_ERS_RESULT_DISCONNECT;
> + }
> +
Looks good to me.
Acked-by: Jacob Keller <jacob.e.keller@intel.com>
Thanks for the bug fix and detailed explanation!
Regards,
Jake
> /* shutdown all operations */
> if (!test_bit(__I40E_SUSPENDED, &pf->state)) {
> rtnl_lock();
next prev parent reply other threads:[~2016-09-27 22:42 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-27 21:14 [PATCH net] i40e: avoid NULL pointer dereference and recursive errors on early PCI error Guilherme G. Piccoli
2016-09-27 22:41 ` Keller, Jacob E [this message]
2016-09-27 23:09 ` [Intel-wired-lan] " Guilherme G. Piccoli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1475016118.22933.1.camel@intel.com \
--to=jacob.e.keller@intel.com \
--cc=gpiccoli@linux.vnet.ibm.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jeffrey.t.kirsher@intel.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).