From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56DBAEB64D9 for ; Mon, 19 Jun 2023 08:22:08 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7889885FD2; Mon, 19 Jun 2023 10:22:05 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=baylibre.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20221208.gappssmtp.com header.i=@baylibre-com.20221208.gappssmtp.com header.b="PGzDHVKo"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id CA169847D3; Mon, 19 Jun 2023 10:22:02 +0200 (CEST) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A490E847D3 for ; Mon, 19 Jun 2023 10:21:58 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=baylibre.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=mkorpershoek@baylibre.com Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-4f7deee339dso3565803e87.0 for ; Mon, 19 Jun 2023 01:21:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20221208.gappssmtp.com; s=20221208; t=1687162918; x=1689754918; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=KSavRc7FqQMB3rR4RK78e/ZRaqFaIf4SD7wfVKRKrog=; b=PGzDHVKoAWl+RPP55BkSTBZiX2de3ACbfMf/ViVOiciXJ6CuBxkBTnvUIyDGXz/4KJ 7B3LjYH2oVAtPWuyyIndNO1abHDg/sHf5LXioiVA3zRpHVBkotnDvlC2zx4buKrxB09i LAaXF0EIz3en3W1NU1QuBSXf4de4bcRCjbSsVBNW+cdP2gHMtQiCzUem4Zuo09mU/OEZ fOKe+J0a42F1HQ5Ua4PqlL8lBHHf7dQHaNZqxpAy59ltMqXvcO8atbreGuBA9BPDubql wenJhx1pTGnM3UUGPmRh3Vd8RsiA5MBO5Y8FjMSKDHZkUaDj5DRUX0zHYKBFtNP96MJB wcOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687162918; x=1689754918; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KSavRc7FqQMB3rR4RK78e/ZRaqFaIf4SD7wfVKRKrog=; b=M4r7Tt6XJ67/RuuYR4G2ShXp1D/GuODQujFHgs88XtUzwwsoO6Z9K9iClC4Wuofhbz TmOT1gVxG6nfPNbfWuUOEeISRFkKoaa7D9jHrACxt6PQHKK7HHvF+gxeZ6nHxkyiFaoX 5eaXBur4WVwaTLr9WPuXIkOAQ6QkTgBd9Mbe6K8B2sx/c0QHQmoFxtoyz+9o24pXvGai M6HvXng8ozhslw6X1BMAzEpya9PRJJOrESWWfrp/Dcb7VJrBZCjlw/SCCpdF/o/bbmVd jY5Mmr6SW8/on1HX/ZNAIu5//EXcPiTNXxjNnup3K93JJ4T0CQYdpHiDj3AOVg73Szbp IyVg== X-Gm-Message-State: AC+VfDwUyb7GSkfpi4nBfCxW3jd0/0QttWLRITnBV+XUeJB/sFySOveJ xtSNzYGmN+MMrzeitc8Hg3SGbA== X-Google-Smtp-Source: ACHHUZ6uas/xAZBsF+l0AWmS9KWwnxPzzq2TASuE6btTLaH9AZvR7xG6SIMDD6sEoH/4jaTVBSPQHQ== X-Received: by 2002:a19:5f18:0:b0:4f8:714e:27a8 with SMTP id t24-20020a195f18000000b004f8714e27a8mr790096lfb.0.1687162916982; Mon, 19 Jun 2023 01:21:56 -0700 (PDT) Received: from localhost ([2a01:cb19:85e6:1900:2bf7:7388:731d:c4e1]) by smtp.gmail.com with ESMTPSA id h25-20020a1ccc19000000b003f42158288dsm9894230wmb.20.2023.06.19.01.21.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Jun 2023 01:21:56 -0700 (PDT) From: Mattijs Korpershoek To: qianfan , Sean Anderson Cc: Guillaume La Roque , Gary Bisson , Troy Kisky , u-boot@lists.denx.de Subject: Re: [PATCH] lib: sparse: allocate blkcnt instead of arbitrary small number In-Reply-To: References: <20230616-sparse-flash-fix-v1-1-6bafeacc567b@baylibre.com> Date: Mon, 19 Jun 2023 10:21:55 +0200 Message-ID: <877crz7tgc.fsf@baylibre.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.8 at phobos.denx.de X-Virus-Status: Clean Hi Qianfan, Thank you for your review. On lun., juin 19, 2023 at 14:19, qianfan wrote: > =E5=9C=A8 2023/6/16 21:26, Mattijs Korpershoek =E5=86=99=E9=81=93: >> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") >> fixed cache alignment for systems with a D-CACHE. >> >> However it introduced some performance regressions [1] on system >> flashing huge images, such as Android. >> >> On AM62x SK EVM, we also observe such performance penalty: >> Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] >> Writing 'super' OKAY [ 75.926s] >> Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] >> Writing 'super' OKAY [ 62.849s] >> Finished. Total time: 182.474s >> >> The reason for this is that we use an arbitrary small buffer >> (info->blksz * 100) for transferring. >> >> Fix it by using a bigger buffer (info->blksz * blkcnt) as suggested in >> the original's patch review [2]. >> >> With this patch, performance impact is mitigated: >> Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.006s] >> Writing 'super' OKAY [ 15.920s] >> Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.651s] >> Writing 'super' OKAY [ 14.665s] >> Finished. Total time: 74.346s >> >> [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@bound= arydevices.com >> [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@s= eco.com/ >> >> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") >> Signed-off-by: Mattijs Korpershoek >> --- >> lib/image-sparse.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/lib/image-sparse.c b/lib/image-sparse.c >> index 5ec0f94ab3eb..25aed0604192 100644 >> --- a/lib/image-sparse.c >> +++ b/lib/image-sparse.c >> @@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_s= torage *info, >> void *data, >> char *response) >> { >> - lbaint_t n =3D blkcnt, write_blks, blks =3D 0, aligned_buf_blks =3D 10= 0; >> + lbaint_t n =3D blkcnt, write_blks, blks =3D 0, aligned_buf_blks =3D bl= kcnt; > Hi: > > It's a good point that this code report the performance was affected by=20 > write large small > mmc blks, not memory copy. I believe memory copy also affects performance, but in my case, it has less impact than small mmc blks. With 62649165cb02 reverted: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.947s] Writing 'super' OKAY [ 12.983s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.600s] Writing 'super' OKAY [ 12.796s] Finished. Total time: 69.430s With aligned_buf_blks =3D blkcnt: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 24.072s] Writing 'super' OKAY [ 16.177s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.681s] Writing 'super' OKAY [ 14.845s] Finished. Total time: 74.919s > > And I can not make sure whether memalign can always alloc such huge=20 > memory when we change the > aligned_buf_blks to blkcnt. Could you clarify the concern here? I've dumped blkcnt for my board (AM62x SK EVK) and the biggest blkcnt I found was: 131072 With info->blksz =3D 512, this gives me: 512 * 131072 =3D 67108864 Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I don't realize it's that much) > > Could you please set aligned_buf_blks to FASTBOOT_MAX_BLK_WRITE(16384)=20 > and test again? With aligned_buf_blks =3D FASTBOOT_MAX_BLK_WRITE(16384): Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super' OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super' OKAY [ 17.192s] Finished. Total time: 76.569s So using FASTBOOT_MAX_BLK_WRITE is slightly worse than using blkcnt. But allocations (for blksz =3D 512) are smaller: 8MB instead of 64MB in my = example. I can spin up a v2 with FASTBOOT_MAX_BLK_WRITE but i'm waiting a little more feedback before doing so. >> uint32_t *aligned_buf =3D NULL; >>=20=20=20 >> if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) { >> >> --- >> base-commit: 2f4664f5c3edc55b18d8906f256a4c8e303243c0 >> change-id: 20230616-sparse-flash-fix-9c2852aa8d16 >> >> Best regards,