From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF76BC4338F for ; Fri, 6 Aug 2021 16:43:19 +0000 (UTC) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D619661163 for ; Fri, 6 Aug 2021 16:43:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D619661163 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.denx.de Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 220F383132; Fri, 6 Aug 2021 18:43:16 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="q5hph8M/"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id C8EB58317A; Fri, 6 Aug 2021 18:43:14 +0200 (CEST) Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 576D680FC5 for ; Fri, 6 Aug 2021 18:43:06 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qv1-xf31.google.com with SMTP id js7so5216326qvb.4 for ; Fri, 06 Aug 2021 09:43:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ncjDyrySqVt6oU35RykCx8ZptfEBiV+aDezynm5BoI0=; b=q5hph8M/+tqmARs72LABabD+msD27+H8kVh1l0haO0X3oFnAU7KB4IAF7W4qwNsZfr kgA+jZbP93vrnWdvi087NEzHVM1dF7PBy+XPQVLGWCEOc+5L+LVo9ANOkQqSLlBVCzSx A8O8gthsB3PdTIVAqiLLQb20uligJqYsVee48= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ncjDyrySqVt6oU35RykCx8ZptfEBiV+aDezynm5BoI0=; b=p8u+dkn0y3OUlObWHNbk2GR8SE+6OFWZamtfsXO66nIU5MqatkaFraO4SzRli2TXeT Ub+qElHGhLK9gPAcoKNC5JWM5W2CGRHetyxqZEhT3lxuhg2KzI/ZbNV3mSKpFfMo98RU PKHA0s9jOP9AE4FrF/PfEqboDtm+omNkwBCdXkMX7A7HQ0/LOWNZha/IIsvwG+DqtD4r dHLR/GfyB20BjNEsMDDZoMYzO9rDTklrmr32Er6dc3QRNllNgU4U/8jFOVxyGLf+tb6B pCZFeNjWLyQP9z6a8k38jbDpxhvnNUCHCiSsYTskvKphlqps8RD5gsNahrEtwTI/avhs znyg== X-Gm-Message-State: AOAM533NoJXVnKP8Y3Y9ZTXCdgU1ilZbymNj3uFqu/BC+9k0CmFP5pM7 lGMvulcUBC/SOhD9420LoZ5i7Q== X-Google-Smtp-Source: ABdhPJwUCTxyPPtN/DID/r+8c/QHT3LQ/dpB6V8PGFtqnB9TqeGk7LuAzS9BGnurkA+hv8Hrpkh8zA== X-Received: by 2002:ad4:5bc7:: with SMTP id t7mr11979719qvt.10.1628268185079; Fri, 06 Aug 2021 09:43:05 -0700 (PDT) Received: from bill-the-cat (2603-6081-7b01-cbda-412a-6045-b0a6-0c94.res6.spectrum.com. [2603:6081:7b01:cbda:412a:6045:b0a6:c94]) by smtp.gmail.com with ESMTPSA id w18sm3573489qtk.6.2021.08.06.09.43.03 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 06 Aug 2021 09:43:04 -0700 (PDT) Date: Fri, 6 Aug 2021 12:43:02 -0400 From: Tom Rini To: Marek Vasut Cc: Jan Kiszka , U-Boot Mailing List , Hai Pham , Simon Goldschmidt , Stephen Warren , Lokesh Vutla Subject: Re: [PATCH] Revert "arm: bootm: Disable LMB reservation for command line and board info on arm64" Message-ID: <20210806164302.GA858@bill-the-cat> References: <7edb5a3f-9896-278d-fe41-e2d48bb8f974@siemens.com> <4602641b-34d2-22dc-01da-e8f46fcc1be3@denx.de> <20210802130421.GS9379@bill-the-cat> <986747ab-71b5-1935-539f-cca506337e2f@siemens.com> <20210802142713.GF9379@bill-the-cat> <20210802144418.GH9379@bill-the-cat> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1ozVqH6bFr/3sC51" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean --1ozVqH6bFr/3sC51 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 05, 2021 at 11:52:05PM +0200, Marek Vasut wrote: > On 8/2/21 4:44 PM, Tom Rini wrote: > > On Mon, Aug 02, 2021 at 04:34:29PM +0200, Jan Kiszka wrote: > > > On 02.08.21 16:27, Tom Rini wrote: > > > > On Mon, Aug 02, 2021 at 04:03:01PM +0200, Jan Kiszka wrote: > > > > > On 02.08.21 15:04, Tom Rini wrote: > > > > > > On Mon, Aug 02, 2021 at 01:54:57PM +0200, Jan Kiszka wrote: > > > > > > > On 02.08.21 13:38, Marek Vasut wrote: > > > > > > > > On 8/2/21 1:36 PM, Jan Kiszka wrote: > > > > > > > > > On 02.08.21 12:48, Marek Vasut wrote: > > > > > > > > > > On 8/2/21 11:37 AM, Jan Kiszka wrote: > > > > > > > > > > > On 02.08.21 02:54, Marek Vasut wrote: > > > > > > > > > > > > On 7/29/21 6:58 PM, Tom Rini wrote: > > > > > > > > > > > >=20 > > > > > > > > > > > > [...] > > > > > > > > > > > >=20 > > > > > > > > > > > > > > > so when did rcar3 introduce something there t= hat shouldn't be > > > > > > > > > > > > > > > reserved?=A0 And you had phrased this to me o= n IRC as about reserving > > > > > > > > > > > > > > > spot > > > > > > > > > > > > > > > for ATAGS, and that not being needed of cours= e on arm64.=A0 But > > > > > > > > > > > > > > > that's > > > > > > > > > > > > > > > not > > > > > > > > > > > > > > > what's going on.=A0 Perhaps the answer is tha= t rcar3 needs to > > > > > > > > > > > > > > > introduce a > > > > > > > > > > > > > > > board_lmb_reserve to free the normal arch one= and provide whatever > > > > > > > > > > > > > > > more > > > > > > > > > > > > > > > narrow scope it needs. > > > > > > > > > > > > > >=20 > > > > > > > > > > > > > > Based on the commit message 2359fa7a878 ("arm: = bootm: Disable LMB > > > > > > > > > > > > > > reservation for command line and board info on = arm64") , this is > > > > > > > > > > > > > > about ATAGS > > > > > > > > > > > > > > and we really don't need to reserve those on ar= m64. > > > > > > > > > > > > >=20 > > > > > > > > > > > > > Commit 2359fa7a878 disables the entire arch_lmb_r= eserve function on > > > > > > > > > > > > > aarch64, yes.=A0 I assumed when we had talked tha= t it was a small area > > > > > > > > > > > > > being set aside and perhaps mis-recalled that ATA= GS tended to live at > > > > > > > > > > > > > DDR_BASE + 0x800 or so. > > > > > > > > > > > >=20 > > > > > > > > > > > > That arch_lmb_reserve() is responsible for reservin= g architecture > > > > > > > > > > > > specific memory. On arm32 it is ATAGS, on arm64 it = is nothing as > > > > > > > > > > > > far as > > > > > > > > > > > > I can tell (and see below regarding the TLB). > > > > > > > > > > > >=20 > > > > > > > > > > > > > This reservation is not at that spot, and a lot > > > > > > > > > > > > > more than that. > > > > > > > > > > > >=20 > > > > > > > > > > > > Can you please elaborate on this "lot more" part ? = Because as much > > > > > > > > > > > > as I > > > > > > > > > > > > studied the reservation code, the "lot more" was AT= AGS on arm32 and > > > > > > > > > > > > nothing on arm64. > > > > > > > > > > >=20 > > > > > > > > > > > See my commit log. > > > > > > > > > >=20 > > > > > > > > > > This is not particularly useful answer, considering the= commit log says: > > > > > > > > > > "lot of crucial things", "Possibly more", "likely also = on other boards" > > > > > > > > > > and other opaque statements. But really, the problem so= far happens on > > > > > > > > > > one K3 board. > > > > > > > > >=20 > > > > > > > > > "Such things are the page table (tlb_addr), > > > > > > > > > relocated U-Boot and the active stack." > > > > > > > >=20 > > > > > > > > Please read the rest of my answer, I don't believe the TLB = should be > > > > > > > > reserved at all. DTTO for the stack. If you think otherwise= , please > > > > > > > > explain why. > > > > > > >=20 > > > > > > > Marek, I've provided you with three generic examples of activ= e memory > > > > > > > blocks that are relevant while U-Boot is allocating from and = also > > > > > > > filling that LMB. Please follow those cases and explain to us= why they > > > > > > > aren't active - or at least prove why they are specific the k= 3 (for > > > > > > > which I found no traces). > > > > > > >=20 > > > > > > > And stop following the TLB topic for now. That was only my fi= rst guess. > > > > > > > The actual crash I'm seeing on my board come from plain code > > > > > > > overwriting. It could have been TLB as well. It could also ha= ve been the > > > > > > > stack. All those become unprotected via your reservation remo= val. > > > > > >=20 > > > > > > Jan, one thing I didn't see before is, are you also using > > > > > > include/configs/ti_armv7_common.h in the end, like the K3 refer= ence > > > > > > platforms, and if not are you setting bootm_size in your enviro= nment? I > > > > > > have one more idea on why this fails on your board but not Mare= k's. > > > > > > Thanks. > > > > >=20 > > > > > We are including that header but we didn't use DEFAULT_LINUX_BOOT= _ENV, > > > > > in fact. That left bootm_size undefined. Can you explain the impa= ct? > > > >=20 > > > > I suspect the answer here is that Marek does not see this problem > > > > because on R-Car bootm_size is set to 0x10000000 and so no relocati= on of > > > > the device tree / kernel / initrd happens to overwrite the running > > > > U-Boot and blow everything up. If you don't revert this, and do set > > > > bootm_size does everything work? Marek, if you unset bootm_size, d= o you > > > > see failure? Thanks! > > > >=20 > > >=20 > > > I currently do not see the error, even with unset bootm_size and Mare= k's > > > patch back in. But fdt indeed moves down when adopting those settings. > > > That makes sense for us anyway, I think our custom env values are rat= her > > > for historic reasons, and one had an issue anyway (incorrect kernel > > > alignment). > > >=20 > > > But at least we understand why I was able to see this, sometimes. > >=20 > > OK, thanks. Note that I'm not sure how I want to move forward here > > because a very frequent user/developer problem is "device tree > > relocated, everything crashed, why? oh, I'll just disable it (and lead > > to another problem down the line)". >=20 > In rcar with bootm_size unset it looks like this: >=20 > =3D> bdinfo > boot_params =3D 0x000000007beee240 > DRAM bank =3D 0x0000000000000000 > -> start =3D 0x0000000048000000 > -> size =3D 0x0000000038000000 > DRAM bank =3D 0x0000000000000001 > -> start =3D 0x0000000500000000 > -> size =3D 0x0000000040000000 > DRAM bank =3D 0x0000000000000002 > -> start =3D 0x0000000600000000 > -> size =3D 0x0000000040000000 > DRAM bank =3D 0x0000000000000003 > -> start =3D 0x0000000700000000 > -> size =3D 0x0000000040000000 > flashstart =3D 0x0000000008000000 > flashsize =3D 0x0000000004000000 > flashoffset =3D 0x00000000000f5890 > baudrate =3D 115200 bps > relocaddr =3D 0x000000007fee8000 > reloc off =3D 0x000000007fee8000 > Build =3D 64-bit > current eth =3D ethernet@e6800000 > ... > fdt_blob =3D 0x000000007beda0e0 > new_fdt =3D 0x000000007beda0e0 > fdt_size =3D 0x000000000000dcc0 > multi_dtb_fit=3D 0x0000000049000000 > lmb_dump_all: > memory.cnt =3D 0x4 > memory[0] [0x48000000-0x7fffffff], 0x38000000 bytes flags: 0 > memory[1] [0x500000000-0x53fffffff], 0x40000000 bytes flags: 0 > memory[2] [0x600000000-0x63fffffff], 0x40000000 bytes flags: 0 > memory[3] [0x700000000-0x73fffffff], 0x40000000 bytes flags: 0 > reserved.cnt =3D 0x1 > reserved[0] [0x44100000-0x47efffff], 0x03e00000 bytes flags: 4 > arch_number =3D 0x0000000000000000 > TLB addr =3D 0x000000007fff0000 > irq_sp =3D 0x000000007beda0d0 > sp start =3D 0x000000007beda0d0 > Early malloc usage: 1318 / 8000 >=20 > ... >=20 > ## Loading kernel from FIT Image at 58000000 ... > Using 'conf-1' configuration > Trying 'kernel-1' kernel subimage > Description: Linux kernel (Sat Jun 5 00:24:15 CEST 2021) > Type: Kernel Image > Compression: uncompressed > Data Start: 0x58000154 > Data Size: 16662536 Bytes =3D 15.9 MiB > Architecture: AArch64 > OS: Linux > Load Address: 0x50200000 > Entry Point: 0x50200000 > Hash algo: crc32 > Hash value: 0655cd1f > Verifying Hash Integrity ... crc32+ OK > ## Loading fdt from FIT Image at 58000000 ... > Using 'conf-1' configuration > Trying 'fdt-1' fdt subimage > Description: Flattened Device Tree blob (Sat Jun 5 00:24:15 CEST > 2021) > Type: Flat Device Tree > Compression: uncompressed > Data Start: 0x58fe42a4 > Data Size: 74686 Bytes =3D 72.9 KiB > Architecture: AArch64 > Hash algo: crc32 > Hash value: 287b2438 > Verifying Hash Integrity ... crc32+ OK > Booting using the fdt blob at 0x58fe42a4 > Loading Kernel Image > Loading Device Tree to 000000007ffea000, end 000000007ffff3bd ... OK OK, I think we can say it's likely that in your case we're relocating the start of the device tree just a bit past where U-Boot is running. A bit of quick math says there's around 1MiB between relocaddr for U-Boot and startof the device tree relocation address. --=20 Tom --1ozVqH6bFr/3sC51 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmENZo8ACgkQFHw5/5Y0 tyzBRQwAgcIZH9DIkMv1Zgt+fzgh8BtI6ettIhy3Rnz+pIVz5gbOFCNGRV7JRyqV QGz4NO9hEzIhofMUWTBQt/nCRMEn6oPURumdOelwD6XczqMeviHSrCyTGWl9T5kg uS7zY6j2Q85e+bK663v8iRrzcZznis8TOlJweASZtBgfQg5pDrlAGAHxn8HgOZhI b3dkZZMbTWsUaAiYqm5q49jBhSyFw6TwZKnqnfJ971Xuf2iBVTtRarGJ9cTM54aH HUG1oxRRvhoRIAWyFqYEXXg4bRihABY6TSxXVq3nzvJvpFnT0QQG5cLWsbJ7nPlL okcdGKkVTvxC/CYPcdcndiKDzx1EFB9sU75r/Vy2d4EyoBIpDXhTmoiKuo4Vq4pS YPu7epVZHxez4UMPOZNrnqZoKO8wCWQOtRUenSJISGRlUV6a5kHG3FT6b7ulx6zs HFN2YwUUsLwakjCOvA7is3SdXErIH3eQIRSWgpjx7mMokuPjIMaBPiy3+gy7KJY5 vdpRgZrP =FiaI -----END PGP SIGNATURE----- --1ozVqH6bFr/3sC51--