From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 648B6C43334 for ; Tue, 14 Jun 2022 15:47:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236671AbiFNPrz (ORCPT ); Tue, 14 Jun 2022 11:47:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243828AbiFNPrx (ORCPT ); Tue, 14 Jun 2022 11:47:53 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 668D013FA8 for ; Tue, 14 Jun 2022 08:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655221669; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c3D5rXBFggKZyiDmUUePVR2BksUY/F8xwEAjtaxtYb8=; b=G/RDSnI7/j1r9uo/0cbVwoYGA7NlGM/El87MQ9VacvYb1hUkeUN2CrPiOPr+WVZ6MXDyfN uKJ/XVRyHtKvnA+7IUge9DQrSM/J2DP78UCcmJ2jXvC98Wb96UK/y61O1UP67EunSJjjXQ ao2Fc0qhwwoeE53gJ+GoRVOf4luR25k= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-118-zUr7boolMWGC-j4ttPm2tg-1; Tue, 14 Jun 2022 11:47:47 -0400 X-MC-Unique: zUr7boolMWGC-j4ttPm2tg-1 Received: by mail-ej1-f72.google.com with SMTP id p10-20020a170906b20a00b0070c21a6d378so3064858ejz.2 for ; Tue, 14 Jun 2022 08:47:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=c3D5rXBFggKZyiDmUUePVR2BksUY/F8xwEAjtaxtYb8=; b=Epy8PNWTvm1japwoZJ16as75V67ZcouUHyZNWERo85Nfx+pnQraYuKMGgyRiGBfAOF mrnwkCLJQ+Ef2daMM4tTNBr3UAcE+fEx+d7wYkLZ6CAZByGnOPjtOKMDqalSmq9Tg9QN b6BRQ9JsB8k+ihZmuf9vLBLYb1RmcJq+r+WW6JA0sHO1hJN21H2p+ff7VsIv1iYpOg/E Y5Teyksa813CNIuVCB4oTVpYueSNdph/gsE9eX3CCu5VYn84xH6+saRaHza9eLceeMnq v50cQvkALa8rCYK9qPJlbQfpVtGzFkyMy8WKAsYWY3GhQQH3ceFCPic7ORD2QLc8QCc0 iRSQ== X-Gm-Message-State: AJIora+zu7uD6IvoXSRzBb0c4Ocx/uq1jvM663CY9zI8d24JBTeN/adC Tek2Q+u+iQGoWuipAtuqXt+2RdAbo/Wy4P5/O0RohxGnWq8u4m5muGiuGlZ3Ti4bozZqB9NCWKk 8/R2VNDPMmzZMuIfT/vsg1g== X-Received: by 2002:a17:907:9496:b0:711:f459:664 with SMTP id dm22-20020a170907949600b00711f4590664mr4822922ejc.704.1655221665108; Tue, 14 Jun 2022 08:47:45 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uL7b2WJ9GqE48Lc8Oa55SIbLe3r72HnMhv92SlzzeuVVgeUFDg0m6ZrYBmGVWItHxA6UV0/w== X-Received: by 2002:a17:907:9496:b0:711:f459:664 with SMTP id dm22-20020a170907949600b00711f4590664mr4822894ejc.704.1655221664848; Tue, 14 Jun 2022 08:47:44 -0700 (PDT) Received: from ?IPV6:2001:1c00:c1e:bf00:d69d:5353:dba5:ee81? (2001-1c00-0c1e-bf00-d69d-5353-dba5-ee81.cable.dynamic.v6.ziggo.nl. [2001:1c00:c1e:bf00:d69d:5353:dba5:ee81]) by smtp.gmail.com with ESMTPSA id u26-20020a170906951a00b00709e786c7b2sm5142116ejx.213.2022.06.14.08.47.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Jun 2022 08:47:44 -0700 (PDT) Message-ID: <5c8923df-3d89-c7be-690d-fbef2c53eeb6@redhat.com> Date: Tue, 14 Jun 2022 17:47:43 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [PATCH] x86/PCI: Revert: "Clip only host bridge windows for E820 regions" Content-Language: en-US To: Bjorn Helgaas Cc: "Rafael J . Wysocki" , Mika Westerberg , =?UTF-8?Q?Krzysztof_Wilczy=c5=84ski?= , Bjorn Helgaas , Myron Stowe , Juha-Pekka Heikkila , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H . Peter Anvin" , =?UTF-8?Q?Benoit_Gr=c3=a9goire?= , Hui Wang , linux-acpi@vger.kernel.org, linux-pci@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, "Guilherme G . Piccoli" References: <20220614151708.GA756100@bhelgaas> From: Hans de Goede In-Reply-To: <20220614151708.GA756100@bhelgaas> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Hi, On 6/14/22 17:17, Bjorn Helgaas wrote: > On Tue, Jun 14, 2022 at 10:15:29AM +0200, Hans de Goede wrote: >> On 6/14/22 01:15, Bjorn Helgaas wrote: >>> On Sun, Jun 12, 2022 at 04:43:25PM +0200, Hans de Goede wrote: >>>> Clipping the bridge windows directly from pci_acpi_root_prepare_resources() >>>> instead of clipping from arch_remove_reservations(), has a number of >>>> unforseen consequences. >>>> >>>> If there is an e820 reservation in the middle of a bridge window, then >>>> the smallest of the 2 remaining parts of the window will be also clipped >>>> off. Where as the previous code would clip regions requested by devices, >>>> rather then the entire window, leaving regions which were either entirely >>>> above or below a reservation in the middle of the window alone. >>>> >>>> E.g. on the Steam Deck this leads to this log message: >>>> >>>> acpi PNP0A08:00: clipped [mem 0x80000000-0xf7ffffff window] to [mem 0xa0100000-0xf7ffffff window] >>>> >>>> which then gets followed by these log messages: >>>> >>>> pci 0000:00:01.2: can't claim BAR 14 [mem 0x80600000-0x806fffff]: no compatible bridge window >>>> pci 0000:00:01.3: can't claim BAR 14 [mem 0x80500000-0x805fffff]: no compatible bridge window >>>> >>>> and many more of these. Ultimately this leads to the Steam Deck >>>> no longer booting properly, so revert the change. >>>> >>>> Note this is not a clean revert, this revert keeps the later change >>>> to make the clipping dependent on a new pci_use_e820 bool, moving >>>> the checking of this bool to arch_remove_reservations(). >>> >>> 4c5e242d3e93 was definitely a mistake (my fault). My intent was to >>> mainly to improve logging of the clipping, but I didn't implement it >>> well. >>> >>> That said, I'd like to understand the connection between the messages >>> you mention and the failure. There are four bridges whose MMIO >>> windows were in the [mem 0x80000000-0x9fffffff] area that we clipped >>> out. The log shows that we moved all those windows and the devices in >>> them to the [mem 0xa0100000-0xf7ffffff] area that remained after >>> clipping. >>> >>> So I think this *should* have worked even though we moved things >>> around unnecessarily. What am I missing? >> >> I don't know? My guess is that maybe the ACPI table do MMIO accesses >> somewhere to hardcoded addresses and moving things breaks the ACPI >> tables. > > This would be a firmware defect, IMHO. There is a mechanism (_DSM for > preserving PCI Boot Configurations) for the firmware to tell us about > things that can't be moved. > >>> The E820 map reports [mem 0xa0000000-0xa00fffff] in the middle of the >>> _CRS, and we currently trim that out. We think this is a firmware >>> defect, so it's likely to break in 2023 if we stop clipping by >>> default. I'm concerned that there may be other things in _CRS that we >>> need to avoid, but firmware isn't telling us about them. >>> >>> Or there's some dependency in the devices that we moved on their >>> original addresses, e.g., firmware on the device latched the address >>> and didn't notice the reassignment. >> >> Right this is the most likely cause I believe. > > This would be another defect, in the device this time. If we can > identify the device, possibly we could quirk around it. > > Either one will be back to bite us in the future if we support > rebalancing resources to make room for hot-added devices. I *think* > Windows already supports this kind of rebalancing. Anyway, this is > why we need to dig a little deeper to figure out exactly what's going > wrong here. Have you looked at the log of the failed boot in the Steam Deck kernel bugzilla? Everything there seems to work just fine and then the system just hangs. I think that maybe it cannot find its root disk, so maybe an NVME issue ? I'm afraid that if we cannot figure out the exact root cause from the failed boot log there is not much more we can do. Unless you have an idea how to debug this further and want to ask the reporter for more info? Regards, Hans