From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexandru Gagniuc Subject: [RFC PATCH v3 2/3] acpi: apei: Do not panic() on PCIe errors reported through GHES Date: Wed, 25 Apr 2018 15:39:50 -0500 Message-ID: <20180425203957.18224-3-mr.nuke.me@gmail.com> References: <20180416215903.7318-1-mr.nuke.me@gmail.com> <20180425203957.18224-1-mr.nuke.me@gmail.com> Return-path: In-Reply-To: <20180425203957.18224-1-mr.nuke.me@gmail.com> Sender: linux-kernel-owner@vger.kernel.org To: linux-acpi@vger.kernel.org, linux-edac@vger.kernel.org Cc: Alexandru Gagniuc , "Rafael J. Wysocki" , Len Brown , Tony Luck , Borislav Petkov , Mauro Carvalho Chehab , Robert Moore , Erik Schmauss , Tyler Baicar , Will Deacon , James Morse , Shiju Jose , "Jonathan (Zhixiong) Zhang" , Dongjiu Geng , linux-kernel@vger.kernel.org, devel@acpica.org List-Id: linux-acpi@vger.kernel.org The policy was to panic() when GHES said that an error is "Fatal". This logic is wrong for several reasons, as it doesn't take into account what caused the error. PCIe fatal errors indicate that the link to a device is either unstable or unusable. They don't indicate that the machine is on fire, and they are not severe enough that we need to panic(). Instead of relying on crackmonkey firmware, evaluate the error severity based on what caused the error (GHES subsections). Signed-off-by: Alexandru Gagniuc --- drivers/acpi/apei/ghes.c | 48 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index f9b53a6f55f3..8ccb9cc10fc8 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -425,8 +425,7 @@ static void ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, int * GHES_SEV_RECOVERABLE -> AER_NONFATAL * GHES_SEV_RECOVERABLE && CPER_SEC_RESET -> AER_FATAL * These both need to be reported and recovered from by the AER driver. - * GHES_SEV_PANIC does not make it to this handling since the kernel must - * panic. + * GHES_SEV_PANIC -> AER_FATAL */ static void ghes_handle_aer(struct acpi_hest_generic_data *gdata) { @@ -459,6 +458,46 @@ static void ghes_handle_aer(struct acpi_hest_generic_data *gdata) #endif } +/* PCIe errors should not cause a panic. */ +static int ghes_sec_pcie_severity(struct acpi_hest_generic_data *gdata) +{ + struct cper_sec_pcie *pcie_err = acpi_hest_get_payload(gdata); + + if (pcie_err->validation_bits & CPER_PCIE_VALID_DEVICE_ID && + pcie_err->validation_bits & CPER_PCIE_VALID_AER_INFO && + IS_ENABLED(CONFIG_ACPI_APEI_PCIEAER)) + return CPER_SEV_RECOVERABLE; + + return ghes_severity(gdata->error_severity); +} +/* + * The severity field in the status block is oftentimes more severe than it + * needs to be. This makes it an unreliable metric for the severity. A more + * reliable way is to look at each subsection and correlate it with how well + * the error can be handled. + * - SEC_PCIE: All PCIe errors can be handled by AER. + */ +static int ghes_actual_severity(struct ghes *ghes) +{ + int worst_sev, sec_sev; + struct acpi_hest_generic_data *gdata; + const guid_t *section_type; + const struct acpi_hest_generic_status *estatus = ghes->estatus; + + worst_sev = GHES_SEV_NO; + apei_estatus_for_each_section(estatus, gdata) { + section_type = (guid_t *)gdata->section_type; + sec_sev = ghes_severity(gdata->error_severity); + + if (guid_equal(section_type, &CPER_SEC_PCIE)) + sec_sev = ghes_sec_pcie_severity(gdata); + + worst_sev = max(worst_sev, sec_sev); + } + + return worst_sev; +} + static void ghes_do_proc(struct ghes *ghes, const struct acpi_hest_generic_status *estatus) { @@ -932,7 +971,7 @@ static void __process_error(struct ghes *ghes) static int ghes_notify_nmi(unsigned int cmd, struct pt_regs *regs) { struct ghes *ghes; - int sev, ret = NMI_DONE; + int sev, asev, ret = NMI_DONE; if (!atomic_add_unless(&ghes_in_nmi, 1, 1)) return ret; @@ -945,8 +984,9 @@ static int ghes_notify_nmi(unsigned int cmd, struct pt_regs *regs) ret = NMI_HANDLED; } + asev = ghes_actual_severity(ghes); sev = ghes_severity(ghes->estatus->error_severity); - if (sev >= GHES_SEV_PANIC) { + if ((sev >= GHES_SEV_PANIC) && (asev >= GHES_SEV_PANIC)) { oops_begin(); ghes_print_queued_estatus(); __ghes_panic(ghes); -- 2.14.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: [RFC,v3,2/3] acpi: apei: Do not panic() on PCIe errors reported through GHES From: Alexandru Gagniuc Message-Id: <20180425203957.18224-3-mr.nuke.me@gmail.com> Date: Wed, 25 Apr 2018 15:39:50 -0500 To: linux-acpi@vger.kernel.org, linux-edac@vger.kernel.org Cc: Alexandru Gagniuc , "Rafael J. Wysocki" , Len Brown , Tony Luck , Borislav Petkov , Mauro Carvalho Chehab , Robert Moore , Erik Schmauss , Tyler Baicar , Will Deacon , James Morse , Shiju Jose , "Jonathan (Zhixiong) Zhang" , Dongjiu Geng , linux-kernel@vger.kernel.org, devel@acpica.org List-ID: VGhlIHBvbGljeSB3YXMgdG8gcGFuaWMoKSB3aGVuIEdIRVMgc2FpZCB0aGF0IGFuIGVycm9yIGlz ICJGYXRhbCIuClRoaXMgbG9naWMgaXMgd3JvbmcgZm9yIHNldmVyYWwgcmVhc29ucywgYXMgaXQg ZG9lc24ndCB0YWtlIGludG8KYWNjb3VudCB3aGF0IGNhdXNlZCB0aGUgZXJyb3IuCgpQQ0llIGZh dGFsIGVycm9ycyBpbmRpY2F0ZSB0aGF0IHRoZSBsaW5rIHRvIGEgZGV2aWNlIGlzIGVpdGhlcgp1 bnN0YWJsZSBvciB1bnVzYWJsZS4gVGhleSBkb24ndCBpbmRpY2F0ZSB0aGF0IHRoZSBtYWNoaW5l IGlzIG9uIGZpcmUsCmFuZCB0aGV5IGFyZSBub3Qgc2V2ZXJlIGVub3VnaCB0aGF0IHdlIG5lZWQg dG8gcGFuaWMoKS4gSW5zdGVhZCBvZgpyZWx5aW5nIG9uIGNyYWNrbW9ua2V5IGZpcm13YXJlLCBl dmFsdWF0ZSB0aGUgZXJyb3Igc2V2ZXJpdHkgYmFzZWQgb24Kd2hhdCBjYXVzZWQgdGhlIGVycm9y IChHSEVTIHN1YnNlY3Rpb25zKS4KClNpZ25lZC1vZmYtYnk6IEFsZXhhbmRydSBHYWduaXVjIDxt ci5udWtlLm1lQGdtYWlsLmNvbT4KLS0tCiBkcml2ZXJzL2FjcGkvYXBlaS9naGVzLmMgfCA0OCAr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0KIDEgZmlsZSBj aGFuZ2VkLCA0NCBpbnNlcnRpb25zKCspLCA0IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2Ry aXZlcnMvYWNwaS9hcGVpL2doZXMuYyBiL2RyaXZlcnMvYWNwaS9hcGVpL2doZXMuYwppbmRleCBm OWI1M2E2ZjU1ZjMuLjhjY2I5Y2MxMGZjOCAxMDA2NDQKLS0tIGEvZHJpdmVycy9hY3BpL2FwZWkv Z2hlcy5jCisrKyBiL2RyaXZlcnMvYWNwaS9hcGVpL2doZXMuYwpAQCAtNDI1LDggKzQyNSw3IEBA IHN0YXRpYyB2b2lkIGdoZXNfaGFuZGxlX21lbW9yeV9mYWlsdXJlKHN0cnVjdCBhY3BpX2hlc3Rf Z2VuZXJpY19kYXRhICpnZGF0YSwgaW50CiAgKiBHSEVTX1NFVl9SRUNPVkVSQUJMRSAtPiBBRVJf Tk9ORkFUQUwKICAqIEdIRVNfU0VWX1JFQ09WRVJBQkxFICYmIENQRVJfU0VDX1JFU0VUIC0+IEFF Ul9GQVRBTAogICogICAgIFRoZXNlIGJvdGggbmVlZCB0byBiZSByZXBvcnRlZCBhbmQgcmVjb3Zl cmVkIGZyb20gYnkgdGhlIEFFUiBkcml2ZXIuCi0gKiBHSEVTX1NFVl9QQU5JQyBkb2VzIG5vdCBt YWtlIGl0IHRvIHRoaXMgaGFuZGxpbmcgc2luY2UgdGhlIGtlcm5lbCBtdXN0Ci0gKiAgICAgcGFu aWMuCisgKiBHSEVTX1NFVl9QQU5JQyAtPiBBRVJfRkFUQUwKICAqLwogc3RhdGljIHZvaWQgZ2hl c19oYW5kbGVfYWVyKHN0cnVjdCBhY3BpX2hlc3RfZ2VuZXJpY19kYXRhICpnZGF0YSkKIHsKQEAg LTQ1OSw2ICs0NTgsNDYgQEAgc3RhdGljIHZvaWQgZ2hlc19oYW5kbGVfYWVyKHN0cnVjdCBhY3Bp X2hlc3RfZ2VuZXJpY19kYXRhICpnZGF0YSkKICNlbmRpZgogfQogCisvKiBQQ0llIGVycm9ycyBz aG91bGQgbm90IGNhdXNlIGEgcGFuaWMuICovCitzdGF0aWMgaW50IGdoZXNfc2VjX3BjaWVfc2V2 ZXJpdHkoc3RydWN0IGFjcGlfaGVzdF9nZW5lcmljX2RhdGEgKmdkYXRhKQoreworCXN0cnVjdCBj cGVyX3NlY19wY2llICpwY2llX2VyciA9IGFjcGlfaGVzdF9nZXRfcGF5bG9hZChnZGF0YSk7CisK KwlpZiAocGNpZV9lcnItPnZhbGlkYXRpb25fYml0cyAmIENQRVJfUENJRV9WQUxJRF9ERVZJQ0Vf SUQgJiYKKwkgICAgcGNpZV9lcnItPnZhbGlkYXRpb25fYml0cyAmIENQRVJfUENJRV9WQUxJRF9B RVJfSU5GTyAmJgorCSAgICBJU19FTkFCTEVEKENPTkZJR19BQ1BJX0FQRUlfUENJRUFFUikpCisJ CXJldHVybiBDUEVSX1NFVl9SRUNPVkVSQUJMRTsKKworCXJldHVybiBnaGVzX3NldmVyaXR5KGdk YXRhLT5lcnJvcl9zZXZlcml0eSk7Cit9CisvKgorICogVGhlIHNldmVyaXR5IGZpZWxkIGluIHRo ZSBzdGF0dXMgYmxvY2sgaXMgb2Z0ZW50aW1lcyBtb3JlIHNldmVyZSB0aGFuIGl0CisgKiBuZWVk cyB0byBiZS4gVGhpcyBtYWtlcyBpdCBhbiB1bnJlbGlhYmxlIG1ldHJpYyBmb3IgdGhlIHNldmVy aXR5LiBBIG1vcmUKKyAqIHJlbGlhYmxlIHdheSBpcyB0byBsb29rIGF0IGVhY2ggc3Vic2VjdGlv biBhbmQgY29ycmVsYXRlIGl0IHdpdGggaG93IHdlbGwKKyAqIHRoZSBlcnJvciBjYW4gYmUgaGFu ZGxlZC4KKyAqICAgLSBTRUNfUENJRTogQWxsIFBDSWUgZXJyb3JzIGNhbiBiZSBoYW5kbGVkIGJ5 IEFFUi4KKyAqLworc3RhdGljIGludCBnaGVzX2FjdHVhbF9zZXZlcml0eShzdHJ1Y3QgZ2hlcyAq Z2hlcykKK3sKKwlpbnQgd29yc3Rfc2V2LCBzZWNfc2V2OworCXN0cnVjdCBhY3BpX2hlc3RfZ2Vu ZXJpY19kYXRhICpnZGF0YTsKKwljb25zdCBndWlkX3QgKnNlY3Rpb25fdHlwZTsKKwljb25zdCBz dHJ1Y3QgYWNwaV9oZXN0X2dlbmVyaWNfc3RhdHVzICplc3RhdHVzID0gZ2hlcy0+ZXN0YXR1czsK KworCXdvcnN0X3NldiA9IEdIRVNfU0VWX05POworCWFwZWlfZXN0YXR1c19mb3JfZWFjaF9zZWN0 aW9uKGVzdGF0dXMsIGdkYXRhKSB7CisJCXNlY3Rpb25fdHlwZSA9IChndWlkX3QgKilnZGF0YS0+ c2VjdGlvbl90eXBlOworCQlzZWNfc2V2ID0gZ2hlc19zZXZlcml0eShnZGF0YS0+ZXJyb3Jfc2V2 ZXJpdHkpOworCisJCWlmIChndWlkX2VxdWFsKHNlY3Rpb25fdHlwZSwgJkNQRVJfU0VDX1BDSUUp KQorCQkJc2VjX3NldiA9IGdoZXNfc2VjX3BjaWVfc2V2ZXJpdHkoZ2RhdGEpOworCisJCXdvcnN0 X3NldiA9IG1heCh3b3JzdF9zZXYsIHNlY19zZXYpOworCX0KKworCXJldHVybiB3b3JzdF9zZXY7 Cit9CisKIHN0YXRpYyB2b2lkIGdoZXNfZG9fcHJvYyhzdHJ1Y3QgZ2hlcyAqZ2hlcywKIAkJCSBj b25zdCBzdHJ1Y3QgYWNwaV9oZXN0X2dlbmVyaWNfc3RhdHVzICplc3RhdHVzKQogewpAQCAtOTMy LDcgKzk3MSw3IEBAIHN0YXRpYyB2b2lkIF9fcHJvY2Vzc19lcnJvcihzdHJ1Y3QgZ2hlcyAqZ2hl cykKIHN0YXRpYyBpbnQgZ2hlc19ub3RpZnlfbm1pKHVuc2lnbmVkIGludCBjbWQsIHN0cnVjdCBw dF9yZWdzICpyZWdzKQogewogCXN0cnVjdCBnaGVzICpnaGVzOwotCWludCBzZXYsIHJldCA9IE5N SV9ET05FOworCWludCBzZXYsIGFzZXYsIHJldCA9IE5NSV9ET05FOwogCiAJaWYgKCFhdG9taWNf YWRkX3VubGVzcygmZ2hlc19pbl9ubWksIDEsIDEpKQogCQlyZXR1cm4gcmV0OwpAQCAtOTQ1LDgg Kzk4NCw5IEBAIHN0YXRpYyBpbnQgZ2hlc19ub3RpZnlfbm1pKHVuc2lnbmVkIGludCBjbWQsIHN0 cnVjdCBwdF9yZWdzICpyZWdzKQogCQkJcmV0ID0gTk1JX0hBTkRMRUQ7CiAJCX0KIAorCQlhc2V2 ID0gZ2hlc19hY3R1YWxfc2V2ZXJpdHkoZ2hlcyk7CiAJCXNldiA9IGdoZXNfc2V2ZXJpdHkoZ2hl cy0+ZXN0YXR1cy0+ZXJyb3Jfc2V2ZXJpdHkpOwotCQlpZiAoc2V2ID49IEdIRVNfU0VWX1BBTklD KSB7CisJCWlmICgoc2V2ID49IEdIRVNfU0VWX1BBTklDKSAmJiAoYXNldiA+PSBHSEVTX1NFVl9Q QU5JQykpIHsKIAkJCW9vcHNfYmVnaW4oKTsKIAkJCWdoZXNfcHJpbnRfcXVldWVkX2VzdGF0dXMo KTsKIAkJCV9fZ2hlc19wYW5pYyhnaGVzKTsK