Re: [docs] [PATCH] dev-manual: disk-space: mention faster "find" command to trim sstate cache

[Michael Opdenacker <michael.opdenacker@bootlin.com>]

Hi Richard,

On 11.08.23 at 17:06, Richard Purdie wrote:
> On Fri, 2023-08-11 at 11:09 +0200, Michael Opdenacker via
> lists.yoctoproject.org wrote:
>> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>>
>> [YOCTO #15182]
>>
>> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>> Reported-by: Yoann CONGAL <yoann.congal@smile.fr>
>> Reported-by: Randy MacLeod <randy.macleod@windriver.com>
>> Reported-by: Josef Holzmayr <jester@theyoctojester.info>
>> ---
>>   documentation/dev-manual/disk-space.rst | 24 +++++++++++++++---------
>>   1 file changed, 15 insertions(+), 9 deletions(-)
>>
>> diff --git a/documentation/dev-manual/disk-space.rst b/documentation/dev-manual/disk-space.rst
>> index c63591cc7a..670f3d2792 100644
>> --- a/documentation/dev-manual/disk-space.rst
>> +++ b/documentation/dev-manual/disk-space.rst
>> @@ -27,19 +27,25 @@ Purging Duplicate Shared State Cache Files
>>   ==========================================
>>   
>>   After multiple build iterations, the Shared State (sstate) cache can contain
>> -duplicate cache files for a given package, while only the most recent one
>> -is likely to be reusable. The following command purges all but the
>> -newest sstate cache file for each package::
>> +duplicate cache files for a given package, consuming a substantial amount of
>> +disk space. However, only the most recent cache files are likeky to be reusable.
>>   
>> -   sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache
>> +The following command is a quick way to purge all the cache files which
>> +are older than a specified number of days::
>>   
>> -This command will ask you to confirm the deletions it identifies.
>> +   find build/sstate-cache -type f -atime +$DAYS -delete
>>   
>> -.. note::
>> +OpenEmbedded-Core also offers a command which can be used to look for
>> +cache files only for enabled architectures, and purge all but the newest
>> +ones on each architecture::
>>   
>> -   The duplicated sstate cache files of one package must have the same
>> -   architecture, which means that sstate cache files with multiple
>> -   architectures are not considered as duplicate.
>> +   sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache
>>   
>> +This command will ask you to confirm the deletions it identifies.
>>   Run ``sstate-cache-management.sh`` for more details about this script.
> As long as bitbake builds have writable access to the sstate cache,
> files are touched as they are accessed so it is easy to know which ones
> are being used and which ones are not. As such you can use mtime in the
> above example even for a partition mounted as "noatime" for performance
> reasons.


Good idea, this way the solution is a little broader as "noatime" is 
frequently used.

>
>>   
>> +.. note::
>> +
>> +   As this command is much more cautious and selective, removing only cache files,
>> +   it will execute much slower than the simple ``find`` command described above.
>> +   Therefore, it may not be your best option to trim huge cache directories.
> It doesn't remove only cache files, it removes files that it considers
> to be unreachable by exploring a set of build configurations. It
> requires full build environment to be available and doesn't work well
> covering multiple releases. It also doesn't work in a limited
> environment like a BSD based NAS where the above find command would
> still likely work easily.
>
> Can we get rid of sstate-cache-management.sh? :)

Thanks for these useful details about this script!

Are you suggesting we could at least remove sstate-cache-management.sh 
from the docs?
Thanks again,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com