[Buildroot] [PATCH 1/1] utils/check-package: handle missing files

[Buildroot] [PATCH 1/1] utils/check-package: handle missing files

[James Knight]

When running check-package before completing commits for a change, if
any files are setup for removal, check-package will throw
FileNotFoundError exceptions instead of generating a warning state. For
example:

 $ utils/docker-run make check-package
 Traceback (most recent call last):
   ...
 FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
 make: *** [Makefile:1264: check-package] Error 1

This commit will now catch FileNotFoundError and populate a warning
message:

 $ utils/docker-run make check-package
 package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
 package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
 427843 lines processed
 3 warnings generated
 make: *** [Makefile:1264: check-package] Error 1

Signed-off-by: James Knight <git@jdknight.me>
---
 utils/check-package | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/utils/check-package b/utils/check-package
index a5ecc2eed8..8781d21d8f 100755
--- a/utils/check-package
+++ b/utils/check-package
@@ -257,18 +257,23 @@ def check_file_using_lib(fname):
         nwarnings += warn
 
     lastline = ""
-    with open(fname, "r", errors="surrogateescape") as f:
-        for lineno, text in enumerate(f):
-            nlines += 1
-            for name, cf in objects:
-                if cf.disable.search(lastline):
-                    continue
-                line_sts = cf.check_line(lineno + 1, text)
-                warn, fail = print_warnings(line_sts, name in xfail)
-                if fail > 0:
-                    failed.add(name)
-                nwarnings += warn
-            lastline = text
+    try:
+        with open(fname, "r", errors="surrogateescape") as f:
+            for lineno, text in enumerate(f):
+                nlines += 1
+                for name, cf in objects:
+                    if cf.disable.search(lastline):
+                        continue
+                    line_sts = cf.check_line(lineno + 1, text)
+                    warn, fail = print_warnings(line_sts, name in xfail)
+                    if fail > 0:
+                        failed.add(name)
+                    nwarnings += warn
+                lastline = text
+    except FileNotFoundError:
+        print(f"{fname}: missing; unstaged file removal?")
+        nwarnings += 1
+        return nwarnings, nlines
 
     for name, cf in objects:
         warn, fail = print_warnings(cf.after(), name in xfail)
-- 
2.49.0.windows.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] utils/check-package: handle missing files

[Romain Naour via buildroot]

Hello James, All,

Le 01/07/2025 à 20:56, James Knight a écrit :
> When running check-package before completing commits for a change, if
> any files are setup for removal, check-package will throw
> FileNotFoundError exceptions instead of generating a warning state. For
> example:
> 
>  $ utils/docker-run make check-package
>  Traceback (most recent call last):
>    ...
>  FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
>  make: *** [Makefile:1264: check-package] Error 1

Indeed, it's due to `git ls-tree -r --name-only HEAD` being used by
check-package Make target:

./utils/check-package `git ls-tree -r --name-only HEAD` \
	--ignore-list=$(TOPDIR)/.checkpackageignore


When working on a specific package, we could use check-package directly without
error:

git rm boot/syslinux/0017-Replace-builtin-strlen-that-appears-to-get-optimized.patch

./utils/check-package  boot/syslinux/*

While running check-package before completing commits for a change is probably
not a good practice, throwing FileNotFoundError exception is not nice :)

Reviewed-by: Romain Naour <romain.naour@smile.fr>

Best regards,
Romain


> 
> This commit will now catch FileNotFoundError and populate a warning
> message:
> 
>  $ utils/docker-run make check-package
>  package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
>  package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
>  427843 lines processed
>  3 warnings generated
>  make: *** [Makefile:1264: check-package] Error 1
> 
> Signed-off-by: James Knight <git@jdknight.me>
> ---
>  utils/check-package | 29 +++++++++++++++++------------
>  1 file changed, 17 insertions(+), 12 deletions(-)
> 
> diff --git a/utils/check-package b/utils/check-package
> index a5ecc2eed8..8781d21d8f 100755
> --- a/utils/check-package
> +++ b/utils/check-package
> @@ -257,18 +257,23 @@ def check_file_using_lib(fname):
>          nwarnings += warn
>  
>      lastline = ""
> -    with open(fname, "r", errors="surrogateescape") as f:
> -        for lineno, text in enumerate(f):
> -            nlines += 1
> -            for name, cf in objects:
> -                if cf.disable.search(lastline):
> -                    continue
> -                line_sts = cf.check_line(lineno + 1, text)
> -                warn, fail = print_warnings(line_sts, name in xfail)
> -                if fail > 0:
> -                    failed.add(name)
> -                nwarnings += warn
> -            lastline = text
> +    try:
> +        with open(fname, "r", errors="surrogateescape") as f:
> +            for lineno, text in enumerate(f):
> +                nlines += 1
> +                for name, cf in objects:
> +                    if cf.disable.search(lastline):
> +                        continue
> +                    line_sts = cf.check_line(lineno + 1, text)
> +                    warn, fail = print_warnings(line_sts, name in xfail)
> +                    if fail > 0:
> +                        failed.add(name)
> +                    nwarnings += warn
> +                lastline = text
> +    except FileNotFoundError:
> +        print(f"{fname}: missing; unstaged file removal?")
> +        nwarnings += 1
> +        return nwarnings, nlines
>  
>      for name, cf in objects:
>          warn, fail = print_warnings(cf.after(), name in xfail)

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] utils/check-package: handle missing files

[Julien Olivain via buildroot]

On 01/07/2025 20:56, James Knight wrote:
> When running check-package before completing commits for a change, if
> any files are setup for removal, check-package will throw
> FileNotFoundError exceptions instead of generating a warning state. For
> example:
> 
>  $ utils/docker-run make check-package
>  Traceback (most recent call last):
>    ...
>  FileNotFoundError: [Errno 2] No such file or directory: 
> 'package/.../0001-some-removed-patch.patch'
>  make: *** [Makefile:1264: check-package] Error 1
> 
> This commit will now catch FileNotFoundError and populate a warning
> message:
> 
>  $ utils/docker-run make check-package
>  package/.../0001-some-removed-patch.patch: missing; unstaged file 
> removal?
>  package/.../0002-another-removed-patch.patch: missing; unstaged file 
> removal?
>  427843 lines processed
>  3 warnings generated
>  make: *** [Makefile:1264: check-package] Error 1
> 
> Signed-off-by: James Knight <git@jdknight.me>

Applied to master, thanks.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] utils/check-package: handle missing files

[Thomas Perale via buildroot]

In reply of:
> When running check-package before completing commits for a change, if
> any files are setup for removal, check-package will throw
> FileNotFoundError exceptions instead of generating a warning state. For
> example:
> 
>  $ utils/docker-run make check-package
>  Traceback (most recent call last):
>    ...
>  FileNotFoundError: [Errno 2] No such file or directory: 'package/.../0001-some-removed-patch.patch'
>  make: *** [Makefile:1264: check-package] Error 1
> 
> This commit will now catch FileNotFoundError and populate a warning
> message:
> 
>  $ utils/docker-run make check-package
>  package/.../0001-some-removed-patch.patch: missing; unstaged file removal?
>  package/.../0002-another-removed-patch.patch: missing; unstaged file removal?
>  427843 lines processed
>  3 warnings generated
>  make: *** [Makefile:1264: check-package] Error 1
> 
> Signed-off-by: James Knight <git@jdknight.me>

Applied to 2025.02.x & 2025.05.x. Thanks

> ---
>  utils/check-package | 29 +++++++++++++++++------------
>  1 file changed, 17 insertions(+), 12 deletions(-)
> 
> diff --git a/utils/check-package b/utils/check-package
> index a5ecc2eed8..8781d21d8f 100755
> --- a/utils/check-package
> +++ b/utils/check-package
> @@ -257,18 +257,23 @@ def check_file_using_lib(fname):
>          nwarnings += warn
>  
>      lastline = ""
> -    with open(fname, "r", errors="surrogateescape") as f:
> -        for lineno, text in enumerate(f):
> -            nlines += 1
> -            for name, cf in objects:
> -                if cf.disable.search(lastline):
> -                    continue
> -                line_sts = cf.check_line(lineno + 1, text)
> -                warn, fail = print_warnings(line_sts, name in xfail)
> -                if fail > 0:
> -                    failed.add(name)
> -                nwarnings += warn
> -            lastline = text
> +    try:
> +        with open(fname, "r", errors="surrogateescape") as f:
> +            for lineno, text in enumerate(f):
> +                nlines += 1
> +                for name, cf in objects:
> +                    if cf.disable.search(lastline):
> +                        continue
> +                    line_sts = cf.check_line(lineno + 1, text)
> +                    warn, fail = print_warnings(line_sts, name in xfail)
> +                    if fail > 0:
> +                        failed.add(name)
> +                    nwarnings += warn
> +                lastline = text
> +    except FileNotFoundError:
> +        print(f"{fname}: missing; unstaged file removal?")
> +        nwarnings += 1
> +        return nwarnings, nlines
>  
>      for name, cf in objects:
>          warn, fail = print_warnings(cf.after(), name in xfail)
> -- 
> 2.49.0.windows.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot