[PATCH] fetch2: Handle missing donestamp file when content is valid

[PATCH] fetch2: Handle missing donestamp file when content is valid

[Nathan Rossi]

In order to allow users to manually populate the download directory with
valid content change the assumption that missing the donestamp file
means unfetched content.

This allows users to populate the download dir without needing to create
dummy .done files such that a user does not need a PREMIRROR when using
BB_NO_NETWORK to provide valid content files in the download directory.

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
---
 lib/bb/fetch2/__init__.py | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 6bd040493e..7f290bb930 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -643,16 +643,13 @@ def verify_donestamp(ud, d, origud=None):
     if not ud.needdonestamp or (origud and not origud.needdonestamp):
         return True
 
-    if not os.path.exists(ud.donestamp):
-        return False
-
     if (not ud.method.supports_checksum(ud) or
         (origud and not origud.method.supports_checksum(origud))):
-        # done stamp exists, checksums not supported; assume the local file is
-        # current
-        return True
+        # if done stamp exists and checksums not supported; assume the local
+        # file is current
+        return os.path.exists(ud.donestamp)
 
-    if not os.path.exists(ud.localpath):
+    if not os.path.exists(ud.localpath) and os.path.exists(ud.donestamp):
         # done stamp exists, but the downloaded file does not; the done stamp
         # must be incorrect, re-trigger the download
         bb.utils.remove(ud.donestamp)
@@ -662,7 +659,7 @@ def verify_donestamp(ud, d, origud=None):
     # Only re-use the precomputed checksums if the donestamp is newer than the
     # file. Do not rely on the mtime of directories, though. If ud.localpath is
     # a directory, there will probably not be any checksums anyway.
-    if (os.path.isdir(ud.localpath) or
+    if os.path.exists(ud.donestamp) and (os.path.isdir(ud.localpath) or
             os.path.getmtime(ud.localpath) < os.path.getmtime(ud.donestamp)):
         try:
             with open(ud.donestamp, "rb") as cachefile:
-- 
2.15.1

Re: [PATCH] fetch2: Handle missing donestamp file when content is valid

[Richard Purdie]

On Thu, 2018-02-01 at 01:07 +1000, Nathan Rossi wrote:
> In order to allow users to manually populate the download directory
> with
> valid content change the assumption that missing the donestamp file
> means unfetched content.
> 
> This allows users to populate the download dir without needing to
> create
> dummy .done files such that a user does not need a PREMIRROR when
> using
> BB_NO_NETWORK to provide valid content files in the download
> directory.
> 
> Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
> ---
>  lib/bb/fetch2/__init__.py | 13 +++++--------
>  1 file changed, 5 insertions(+), 8 deletions(-)

Could I persuade you into adding a test of this in
lib/bb/tests/fetch.py?

The advantage is that then we hopefully don't break this again!

Cheers,

Richard

Re: [PATCH] fetch2: Handle missing donestamp file when content is valid

[Nathan Rossi]

On 1 February 2018 at 23:12, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2018-02-01 at 01:07 +1000, Nathan Rossi wrote:
>> In order to allow users to manually populate the download directory
>> with
>> valid content change the assumption that missing the donestamp file
>> means unfetched content.
>>
>> This allows users to populate the download dir without needing to
>> create
>> dummy .done files such that a user does not need a PREMIRROR when
>> using
>> BB_NO_NETWORK to provide valid content files in the download
>> directory.
>>
>> Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
>> ---
>>  lib/bb/fetch2/__init__.py | 13 +++++--------
>>  1 file changed, 5 insertions(+), 8 deletions(-)
>
> Could I persuade you into adding a test of this in
> lib/bb/tests/fetch.py?
>
> The advantage is that then we hopefully don't break this again!

Sure, I'll sort out a v2 with some tests.

Regards,
Nathan