Re: [PATCH] lib: sparse: allocate blkcnt instead of arbitrary small number

[Mattijs Korpershoek <mkorpershoek@baylibre.com>]

Hi Qianfan,

Thank you for your review.

On lun., juin 19, 2023 at 14:19, qianfan <qianfanguijin@163.com> wrote:

> 在 2023/6/16 21:26, Mattijs Korpershoek 写道:
>> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
>> fixed cache alignment for systems with a D-CACHE.
>>
>> However it introduced some performance regressions [1] on system
>> flashing huge images, such as Android.
>>
>> On AM62x SK EVM, we also observe such performance penalty:
>> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.954s]
>> Writing 'super'                                    OKAY [ 75.926s]
>> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.641s]
>> Writing 'super'                                    OKAY [ 62.849s]
>> Finished. Total time: 182.474s
>>
>> The reason for this is that we use an arbitrary small buffer
>> (info->blksz * 100) for transferring.
>>
>> Fix it by using a bigger buffer (info->blksz * blkcnt) as suggested in
>> the original's patch review [2].
>>
>> With this patch, performance impact is mitigated:
>> Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 24.006s]
>> Writing 'super'                                    OKAY [ 15.920s]
>> Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.651s]
>> Writing 'super'                                    OKAY [ 14.665s]
>> Finished. Total time: 74.346s
>>
>> [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bisson@boundarydevices.com
>> [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569bd18@seco.com/
>>
>> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned")
>> Signed-off-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
>> ---
>>   lib/image-sparse.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/lib/image-sparse.c b/lib/image-sparse.c
>> index 5ec0f94ab3eb..25aed0604192 100644
>> --- a/lib/image-sparse.c
>> +++ b/lib/image-sparse.c
>> @@ -55,7 +55,7 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info,
>>   				       void *data,
>>   				       char *response)
>>   {
>> -	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100;
>> +	lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = blkcnt;
> Hi:
>
> It's a good point that this code report the performance was affected by 
> write large small
> mmc blks, not memory copy.

I believe memory copy also affects performance, but in my case,
it has less impact than small mmc blks.

With 62649165cb02 reverted:
Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.947s]
Writing 'super'                                    OKAY [ 12.983s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.600s]
Writing 'super'                                    OKAY [ 12.796s]
Finished. Total time: 69.430s

With aligned_buf_blks = blkcnt:
Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 24.072s]
Writing 'super'                                    OKAY [ 16.177s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.681s]
Writing 'super'                                    OKAY [ 14.845s]
Finished. Total time: 74.919s

>
> And I can not make sure whether memalign can always alloc such huge 
> memory when we change the
> aligned_buf_blks to blkcnt.

Could you clarify the concern here? I've dumped blkcnt for my board
(AM62x SK EVK) and the biggest blkcnt I found was: 131072

With info->blksz = 512, this gives me: 512 * 131072 = 67108864

Which is a memalign (memory alloc) of 64MB. Is 64MB really that big? (I
don't realize it's that much)

>
> Could you please set aligned_buf_blks to FASTBOOT_MAX_BLK_WRITE(16384) 
> and test again?

With aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE(16384):
Sending sparse 'super' 1/2 (768793 KB)             OKAY [ 23.912s]
Writing 'super'                                    OKAY [ 15.780s]
Sending sparse 'super' 2/2 (629819 KB)             OKAY [ 19.581s]
Writing 'super'                                    OKAY [ 17.192s]
Finished. Total time: 76.569s

So using FASTBOOT_MAX_BLK_WRITE is slightly worse than using blkcnt.
But allocations (for blksz = 512) are smaller: 8MB instead of 64MB in my example.

I can spin up a v2 with FASTBOOT_MAX_BLK_WRITE but i'm waiting a little
more feedback before doing so.

>>   	uint32_t *aligned_buf = NULL;
>>   
>>   	if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) {
>>
>> ---
>> base-commit: 2f4664f5c3edc55b18d8906f256a4c8e303243c0
>> change-id: 20230616-sparse-flash-fix-9c2852aa8d16
>>
>> Best regards,