[LTP] [PATCH] doc: fix stats page

[LTP] [PATCH] doc: fix stats page

[Andrea Cervesato]

From: Andrea Cervesato <andrea.cervesato@suse.com>

In the last documentation builds we faced syscalls file download error.
In particular, git.kernel.org communication started to fail with 403.
The reason seems to be realted to the server configuration, that is now
requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid
this issue.

Signed-off-by: Andrea Cervesato <andrea.cervesato@suse.com>
---
Fix the following error during 'make':

	WARNING: Can't download syscall_n64.tbl from kernel sources (HTTP Error 403: Forbidden)
---
 doc/conf.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index 8cc890fcc8d0650aac82adabbbd434c55c7b8ad3..23fe7a1b9742f604a5b97f0db62b29ed9c64c445 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -126,8 +126,18 @@ def generate_syscalls_stats(_):
     error = False
     try:
         socket.setdefaulttimeout(3)
-        urllib.request.urlretrieve(
-            f"{syscalls_url}/syscall_n64.tbl", "syscalls.tbl")
+
+        # kernel.org doesn't always allow to download syscalls file, so we need
+        # to emulate a different client in order to download it. Browser
+        # emulation might fail due to captcha request and for this reason we
+        # use the 'curl' command instead
+        req = urllib.request.Request(
+            f"{syscalls_url}/syscall_n64.tbl",
+            headers={'User-Agent': 'curl/8.6.0'})
+
+        with urllib.request.urlopen(req) as response:
+            with open("syscalls.tbl", 'wb') as out_file:
+                out_file.write(response.read())
     except urllib.error.URLError as err:
         error = True
         logger = sphinx.util.logging.getLogger(__name__)

---
base-commit: 8e0d3347944b2044a198ddd69d4be186fcda65ef
change-id: 20250416-fix_docs_stats-460880da49e6

Best regards,
-- 
Andrea Cervesato <andrea.cervesato@suse.com>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] doc: fix stats page

[Petr Vorel]

Hi Andrea,

> In the last documentation builds we faced syscalls file download error.
> In particular, git.kernel.org communication started to fail with 403.
> The reason seems to be realted to the server configuration, that is now
> requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid
> this issue.

Thanks for fixing this! I also noticed this, was in my TODO.
I also prefer to keep using korg if possible than start using
github/torvalds/linux (which is not official, might be outdated or also limited
similar way for bots).

IMHO this is part of the changes in the korg infrastructure detecting bots.
I guess they need to reduce the traffic.

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH] doc: fix stats page

[Andrea Cervesato via ltp]

Pushed thanks.

- Andrea

On 4/16/25 14:30, Petr Vorel wrote:
> Hi Andrea,
>
>> In the last documentation builds we faced syscalls file download error.
>> In particular, git.kernel.org communication started to fail with 403.
>> The reason seems to be realted to the server configuration, that is now
>> requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid
>> this issue.
> Thanks for fixing this! I also noticed this, was in my TODO.
> I also prefer to keep using korg if possible than start using
> github/torvalds/linux (which is not official, might be outdated or also limited
> similar way for bots).
>
> IMHO this is part of the changes in the korg infrastructure detecting bots.
> I guess they need to reduce the traffic.
>
> Reviewed-by: Petr Vorel <pvorel@suse.cz>
>
> Kind regards,
> Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp