linux-acpi.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Alexandru Gagniuc <mr.nuke.me@gmail.com>
To: linux-acpi@vger.kernel.org, linux-edac@vger.kernel.org
Cc: rjw@rjwysocki.net, lenb@kernel.org, tony.luck@intel.com,
	bp@alien8.de, tbaicar@codeaurora.org, will.deacon@arm.com,
	james.morse@arm.com, shiju.jose@huawei.com,
	zjzhang@codeaurora.org, gengdongjiu@huawei.com,
	linux-kernel@vger.kernel.org, alex_gagniuc@dellteam.com,
	austin_bolen@dell.com, shyam_iyer@dell.com, devel@acpica.org,
	mchehab@kernel.org, robert.moore@intel.com,
	erik.schmauss@intel.com, Alexandru Gagniuc <mr.nuke.me@gmail.com>
Subject: [RFC PATCH v2 3/4] acpi: apei: Do not panic() when correctable errors are marked as fatal.
Date: Mon, 16 Apr 2018 16:59:02 -0500	[thread overview]
Message-ID: <20180416215903.7318-4-mr.nuke.me@gmail.com> (raw)
In-Reply-To: <20180416215903.7318-1-mr.nuke.me@gmail.com>

Firmware is evil:
 - ACPI was created to "try and make the 'ACPI' extensions somehow
 Windows specific" in order to "work well with NT and not the others
 even if they are open"
 - EFI was created to hide "secret" registers from the OS.
 - UEFI was created to allow compromising an otherwise secure OS.

Never has firmware been created to solve a problem or simplify an
otherwise cumbersome process. It is of no surprise then, that
firmware nowadays intentionally crashes an OS.

One simple way to do that is to mark GHES errors as fatal. Firmware
knows and even expects that an OS will crash in this case. And most
OSes do.

PCIe errors are notorious for having different definitions of "fatal".
In ACPI, and other firmware sandards, 'fatal' means the machine is
about to explode and needs to be reset. In PCIe, on the other hand,
fatal means that the link to a device has died. In the hotplug world
of PCIe, this is akin to a USB disconnect. From that view, the "fatal"
loss of a link is a normal event. To allow a machine to crash in this
case is downright idiotic.

To solve this, implement an IRQ safe handler for AER. This makes sure
we have enough information to invoke the full AER handler later down
the road, and tells ghes_notify_nmi that "It's all cool".
ghes_notify_nmi() then gets calmed down a little, and doesn't panic().

Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
---
 drivers/acpi/apei/ghes.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 42 insertions(+), 2 deletions(-)

diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
index 2119c51b4a9e..e0528da4e8f8 100644
--- a/drivers/acpi/apei/ghes.c
+++ b/drivers/acpi/apei/ghes.c
@@ -481,12 +481,26 @@ static int ghes_handle_aer(struct acpi_hest_generic_data *gdata, int sev)
 	return ghes_severity(gdata->error_severity);
 }
 
+static int ghes_handle_aer_irqsafe(struct acpi_hest_generic_data *gdata,
+				   int sev)
+{
+	struct cper_sec_pcie *pcie_err = acpi_hest_get_payload(gdata);
+
+	/* The system can always recover from AER errors. */
+	if (pcie_err->validation_bits & CPER_PCIE_VALID_DEVICE_ID &&
+		pcie_err->validation_bits & CPER_PCIE_VALID_AER_INFO)
+		return CPER_SEV_RECOVERABLE;
+
+	return ghes_severity(gdata->error_severity);
+}
+
 /**
  * ghes_handler - handler for ACPI APEI errors
  * @error_uuid: UUID describing the error entry (See ACPI/EFI CPER for details)
  * @handle: Handler for the GHES entry of type 'error_uuid'. The handler
  *	returns the severity of the error after handling. A handler is allowed
  *	to demote errors to correctable or corrected, as appropriate.
+ * @handle_irqsafe: (optional) Non-blocking handler for GHES entry.
  */
 static const struct ghes_handler {
 	const guid_t *error_uuid;
@@ -498,6 +512,7 @@ static const struct ghes_handler {
 		.handle = ghes_handle_mem,
 	}, {
 		.error_uuid = &CPER_SEC_PCIE,
+		.handle_irqsafe = ghes_handle_aer_irqsafe,
 		.handle = ghes_handle_aer,
 	}, {
 		.error_uuid = &CPER_SEC_PROC_ARM,
@@ -551,6 +566,30 @@ static void ghes_do_proc(struct ghes *ghes,
 	}
 }
 
+/* How severe is the error if handling is deferred outside IRQ/NMI context? */
+static int ghes_deferrable_severity(struct ghes *ghes)
+{
+	int deferrable_sev, sev, sec_sev;
+	struct acpi_hest_generic_data *gdata;
+	const struct ghes_handler *handler;
+	const guid_t *section_type;
+	const struct acpi_hest_generic_status *estatus = ghes->estatus;
+
+	deferrable_sev = GHES_SEV_NO;
+	sev = ghes_severity(estatus->error_severity);
+	apei_estatus_for_each_section(estatus, gdata) {
+		section_type = (guid_t *)gdata->section_type;
+		handler = get_handler(section_type);
+		if (handler && handler->handle_irqsafe)
+			sec_sev = handler->handle_irqsafe(gdata, sev);
+		else
+			sec_sev = ghes_severity(gdata->error_severity);
+		deferrable_sev = max(deferrable_sev, sec_sev);
+	}
+
+	return deferrable_sev;
+}
+
 static void __ghes_print_estatus(const char *pfx,
 				 const struct acpi_hest_generic *generic,
 				 const struct acpi_hest_generic_status *estatus)
@@ -980,7 +1019,7 @@ static void __process_error(struct ghes *ghes)
 static int ghes_notify_nmi(unsigned int cmd, struct pt_regs *regs)
 {
 	struct ghes *ghes;
-	int sev, ret = NMI_DONE;
+	int sev, dsev, ret = NMI_DONE;
 
 	if (!atomic_add_unless(&ghes_in_nmi, 1, 1))
 		return ret;
@@ -993,8 +1032,9 @@ static int ghes_notify_nmi(unsigned int cmd, struct pt_regs *regs)
 			ret = NMI_HANDLED;
 		}
 
+		dsev = ghes_deferrable_severity(ghes);
 		sev = ghes_severity(ghes->estatus->error_severity);
-		if (sev >= GHES_SEV_PANIC) {
+		if ((sev >= GHES_SEV_PANIC) && (dsev >= GHES_SEV_PANIC)) {
 			oops_begin();
 			ghes_print_queued_estatus();
 			__ghes_panic(ghes);
-- 
2.14.3

  parent reply	other threads:[~2018-04-16 21:59 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-16 21:58 [RFC PATCH v2 0/4] acpi: apei: Improve error handling with firmware-first Alexandru Gagniuc
2018-04-16 21:59 ` [RFC PATCH v2 1/4] EDAC, GHES: Remove unused argument to ghes_edac_report_mem_error Alexandru Gagniuc
2018-04-17  9:36   ` Borislav Petkov
2018-04-17 16:43     ` Alex G.
2018-04-16 21:59 ` [RFC PATCH v2 2/4] acpi: apei: Split GHES handlers outside of ghes_do_proc Alexandru Gagniuc
2018-04-18 17:52   ` Borislav Petkov
2018-04-19 14:19     ` Alex G.
2018-04-19 14:30       ` Borislav Petkov
2018-04-19 14:57         ` Alex G.
2018-04-19 15:29           ` Borislav Petkov
2018-04-19 15:46             ` Alex G.
2018-04-19 16:40               ` Borislav Petkov
2018-04-16 21:59 ` Alexandru Gagniuc [this message]
2018-04-18 17:54   ` [RFC PATCH v2 3/4] acpi: apei: Do not panic() when correctable errors are marked as fatal Borislav Petkov
2018-04-19 14:57     ` Alex G.
2018-04-19 15:35       ` James Morse
2018-04-19 16:27         ` Alex G.
2018-04-19 15:40       ` Borislav Petkov
2018-04-19 16:26         ` Alex G.
2018-04-19 16:45           ` Borislav Petkov
2018-04-19 17:40             ` Alex G.
2018-04-19 19:03               ` Borislav Petkov
2018-04-19 22:55                 ` Alex G.
2018-04-22 10:48                   ` Borislav Petkov
2018-04-24  4:19                     ` Alex G.
2018-04-25 14:01                       ` Borislav Petkov
2018-04-25 15:00                         ` Alex G.
2018-04-25 17:15                           ` Borislav Petkov
2018-04-25 17:27                             ` Alex G.
2018-04-25 17:39                               ` Borislav Petkov
2018-04-16 21:59 ` [RFC PATCH v2 4/4] acpi: apei: Warn when GHES marks correctable errors as "fatal" Alexandru Gagniuc
2018-04-18 17:54   ` Borislav Petkov
2018-04-19 15:11     ` Alex G.
2018-04-19 15:46       ` Borislav Petkov
2018-04-25 20:39 ` [RFC PATCH v3 0/3] acpi: apei: Improve PCIe error handling with firmware-first Alexandru Gagniuc
2018-04-25 20:39   ` [RFC PATCH v3 1/3] EDAC, GHES: Remove unused argument to ghes_edac_report_mem_error Alexandru Gagniuc
2018-04-25 20:39   ` [RFC PATCH v3 2/3] acpi: apei: Do not panic() on PCIe errors reported through GHES Alexandru Gagniuc
2018-04-26 11:19     ` Borislav Petkov
2018-04-26 17:44       ` Alex G.
2018-04-25 20:39   ` [RFC PATCH v3 3/3] acpi: apei: Warn when GHES marks correctable errors as "fatal" Alexandru Gagniuc
2018-04-26 11:20     ` Borislav Petkov
2018-04-26 17:47       ` Alex G.
2018-04-26 18:03         ` Borislav Petkov
2018-05-02 19:10       ` Pavel Machek
2018-05-02 19:29         ` Alex G.

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180416215903.7318-4-mr.nuke.me@gmail.com \
    --to=mr.nuke.me@gmail.com \
    --cc=alex_gagniuc@dellteam.com \
    --cc=austin_bolen@dell.com \
    --cc=bp@alien8.de \
    --cc=devel@acpica.org \
    --cc=erik.schmauss@intel.com \
    --cc=gengdongjiu@huawei.com \
    --cc=james.morse@arm.com \
    --cc=lenb@kernel.org \
    --cc=linux-acpi@vger.kernel.org \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchehab@kernel.org \
    --cc=rjw@rjwysocki.net \
    --cc=robert.moore@intel.com \
    --cc=shiju.jose@huawei.com \
    --cc=shyam_iyer@dell.com \
    --cc=tbaicar@codeaurora.org \
    --cc=tony.luck@intel.com \
    --cc=will.deacon@arm.com \
    --cc=zjzhang@codeaurora.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).