[PATCH 0/4] fetch2: add alternative fetch method based on curl

[PATCH 0/4] fetch2: add alternative fetch method based on curl

[Pascal Eberhard via B4 Relay]

wget fetch method is used for web downloads with http, https, ftp, ftps
protocols. wget cmdline tool is missing some features such as hostname
resolution by the proxy when using SOCKS5 proxy. SSH tunnel provides
this feature for example.

This curl fetch method is based on curl cmdline tool and provides the
same protocols as wget and supports SOCKS5 hostname resolution.
This class inherits wget method in order to avoid code duplication as
much as possible.

wget remains the default download method. curl fetch method can be
enabled by setting a new bitbake variable:
  BB_FETCH_METHOD_HTTP = "curl"

The hostname resolution by SOCKS5 proxy is activated by setting
environment variable:
  all_proxy="socks5h://...""

a patch on ce-core will be sent if this proposal is accepted to
conditionnaly add curl to HOSTTOOLS variable in meta/conf/bitbake.conf:
HOSTTOOLS += "${@bb.utils.contains('BB_FETCH_METHOD_HTTP', 'curl', 'curl','', d)}"

Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
---
Pascal Eberhard (4):
      fetch2: add curl method to fetch web content
      fetch2: make curl method activable with BB_FETCH_METHOD_HTTP
      lib/tests/fetch2: add tests for curl method
      doc: bitbake-user-manual-ref-variables: describe BB_FETCH_METHOD_HTTP variable

 .../bitbake-user-manual-ref-variables.rst          |   5 +
 lib/bb/fetch2/__init__.py                          |   2 +
 lib/bb/fetch2/curl.py                              | 162 +++++++++++++++++++++
 lib/bb/fetch2/wget.py                              |  10 ++
 lib/bb/tests/fetch.py                              |  68 +++++++++
 5 files changed, 247 insertions(+)
---
base-commit: 48efc36b4e03f736e7521d269ced3417522784e9
change-id: 20260305-add_alt_fetch_method_curl-7a0bf96ad223

Best regards,
-- 
Pascal Eberhard <pascal.eberhard@se.com>

[PATCH 1/4] fetch2: add curl method to fetch web content

[Pascal Eberhard via B4 Relay]

From: Pascal Eberhard <pascal.eberhard@se.com>

curl fetch method is an alternative fetch method for web downloads. It
is based on curl cmdline tool and provides the same http, https, ftp
and ftps protocols as wget. It supports some new features as well such
as hostname resolution by the proxy when using SOCKS5 proxy.

Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
---
 lib/bb/fetch2/curl.py | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 162 insertions(+)

diff --git a/lib/bb/fetch2/curl.py b/lib/bb/fetch2/curl.py
new file mode 100644
index 000000000..250805233
--- /dev/null
+++ b/lib/bb/fetch2/curl.py
@@ -0,0 +1,162 @@
+"""
+BitBake 'Fetch' implementations for web downloads based on curl.
+
+curl fetch method is an alternative to existing wget method and can be enabled
+by setting bitbake variable:
+  BB_FETCH_METHOD_HTTP = "curl"
+
+curl fetch method provides new features such as hostname resolution by the
+proxy itself when using SOCKS5 proxy. It can be set with environment variable:
+  all_proxy="socks5h://...""
+"""
+
+# Copyright (C) 2026, Schneider Electric
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Based on the wget fetcher method, Copyright 2003 Holger Schurig
+
+import os
+import re
+import shlex
+import tempfile
+
+import bb
+import bb.fetch2
+import bb.progress
+import bb.utils
+from bb.data_smart import DataSmart
+from bb.fetch2 import FetchData, FetchError, logger, runfetchcmd
+from bb.fetch2.wget import Wget
+
+
+class CurlProgressHandler(bb.progress.LineFilterProgressHandler):
+    """
+    Extract progress information from curl commandline output.
+    Note: relies on --progress-bar being specified on the curl command line.
+    """
+
+    def __init__(self, d: DataSmart):
+        super(CurlProgressHandler, self).__init__(d)
+        # Send an initial progress event so the bar gets shown
+        self._fire_progress(0)
+
+    def writeline(self, line: str):
+        matches = re.findall(r' ([\d]+)\.\d%', line)
+        if matches:
+            progress = int(matches[0])
+            self.update(progress)
+            return False
+        return True
+
+
+class Curl(Wget):
+    """
+    Class to fetch urls via curl cmdline tool.
+    The code not related to the cmdline is the same between wget and curl.
+    Curl class inherits Wget class to avoid code duplication.
+    """
+
+    def is_enabled(self, d) -> bool:
+        """
+        curl method is enabled when BB_FETCH_METHOD_HTTP = "curl" only.
+        """
+        method_http: str = d.getVar("BB_FETCH_METHOD_HTTP")
+        return method_http == "curl"
+
+    def supports(self, ud: FetchData, d: DataSmart) -> bool:
+        """
+        Check if a given url can be fetched with curl.
+        """
+        if not self.is_enabled(d):
+            return False
+        if ud.type not in ['http', 'https', 'ftp', 'ftps']:
+            return False
+        logger.debug2("Fetch method 'curl' enabled")
+        return True
+
+    def urldata_init(self, ud: FetchData, d: DataSmart):
+        if 'protocol' in ud.parm:
+            if ud.parm['protocol'] == 'git':
+                raise bb.fetch2.ParameterError("Invalid protocol - if you wish to fetch from a git repository using http, you need to instead use the git:// prefix with protocol=http", ud.url)
+
+        if 'downloadfilename' in ud.parm:
+            ud.basename: str = ud.parm['downloadfilename']
+        else:
+            ud.basename: str = os.path.basename(ud.path)
+
+        ud.localfile = ud.basename
+        if not ud.localfile:
+            ud.localfile = ud.host + ud.path.replace("/", ".")
+
+        # --retry 1: equivalent to --tries=2 of wget.
+        # --speed-limit 1 --speed-time 100 --connect-timeout 100: equivalent to --timeout=100 option of wget.
+        # --location: redo request on new location when a page as moved, indicated with 3xx response code.
+        # --fail: fails with exit code when server generates HTML error rather than writing HTML error to output.
+        self.basecmd: str = d.getVar("FETCHCMD_curl") or "/usr/bin/env curl --retry 1 --speed-limit 1 --speed-time 100 --connect-timeout 100 --location --fail"
+
+        if ud.type == 'ftp' or ud.type == 'ftps':
+            self.basecmd += " --ftp-pasv"
+
+        if not self.check_certs(d):
+            self.basecmd += " --insecure"
+
+    def _runcurl(self, ud: FetchData, d: DataSmart, command: str, quiet: bool, workdir: str | None = None):
+        progresshandler = CurlProgressHandler(d)
+
+        logger.debug2("Fetching %s using command '%s'" % (ud.url, command))
+        bb.fetch2.check_network_access(d, command, ud.url)
+        runfetchcmd(command + " --progress-bar", d, quiet, log=progresshandler, workdir=workdir)
+
+    def download(self, ud: FetchData, d: DataSmart):
+        """Fetch urls"""
+        fetchcmd: str = self.basecmd
+        dldir: str = os.path.realpath(d.getVar("DL_DIR"))
+        localpath: str = os.path.join(dldir, ud.localfile) + ".tmp"
+        bb.utils.mkdirhier(os.path.dirname(localpath))
+        fetchcmd += " --output %s" % shlex.quote(localpath)
+
+        if ud.user and ud.pswd:
+            fetchcmd += " --anyauth"
+            if ud.parm.get("redirectauth", "1") == "1":
+                fetchcmd += f" --user={ud.user}:{ud.pswd}"
+
+        uri: str = ud.url.split(";")[0]
+        fetchcmd += f" --continue-at - '{uri}'"
+
+        self._runcurl(ud, d, fetchcmd, False)
+
+        # Sanity check since curl can pretend it succeed when it didn't
+        # Also, this used to happen if sourceforge sent us to the mirror page
+        if not os.path.exists(localpath):
+            raise FetchError(f"The fetch command returned success for url {uri} but {localpath} doesn't exist?!", uri)
+
+        if os.path.getsize(localpath) == 0:
+            os.remove(localpath)
+            raise FetchError(f"The fetch of {uri} resulted in a zero size file?! Deleting and failing since this isn't right.", uri)
+
+        # Try and verify any checksum now, meaning if it isn't correct, we don't remove the
+        # original file, which might be a race (imagine two recipes referencing the same
+        # source, one with an incorrect checksum)
+        bb.fetch2.verify_checksum(ud, d, localpath=localpath, fatal_nochecksum=False)
+
+        # Remove the ".tmp" and move the file into position atomically
+        # Our lock prevents multiple writers but mirroring code may grab incomplete files
+        os.rename(localpath, localpath[:-4])
+
+        return True
+
+    def _fetch_index(self, uri: str, ud: FetchData, d: DataSmart):
+        """
+        Run fetch checkstatus to get directory information
+        """
+        with tempfile.TemporaryDirectory(prefix="curl-index-") as workdir, tempfile.NamedTemporaryFile(dir=workdir, prefix="curl-listing-") as f:
+            fetchcmd: str = self.basecmd
+            fetchcmd += f" --output {f.name} '{uri}'"
+            try:
+                self._runcurl(ud, d, fetchcmd, True, workdir=workdir)
+                fetchresult = f.read()
+            except bb.fetch2.BBFetchException:
+                fetchresult = ""
+
+        return fetchresult

-- 
2.43.0

Re: [bitbake-devel] [PATCH 1/4] fetch2: add curl method to fetch web content

[Mathieu Dubois-Briand]

On Thu Mar 5, 2026 at 4:32 PM CET, Pascal Eberhard via B4 Relay via lists.openembedded.org wrote:
> From: Pascal Eberhard <pascal.eberhard@se.com>
>
> curl fetch method is an alternative fetch method for web downloads. It
> is based on curl cmdline tool and provides the same http, https, ftp
> and ftps protocols as wget. It supports some new features as well such
> as hostname resolution by the proxy when using SOCKS5 proxy.
>
> Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
> ---

Hi Pascal,

Thanks for your patches.

> +
> +    def _runcurl(self, ud: FetchData, d: DataSmart, command: str, quiet: bool, workdir: str | None = None):

Writing type hint unions with a pipe was added recently to Python, 3.10
if I'm correct [1]. This is above the minimal 3.9 [2] version mandated
for bitbake.

And this is indeed failing on some supported distributions, at least on
Debian 11 and Rocky 9:

Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/genericarm64/build/repos/bitbake/bin/bitbake-setup", line 27, in <module>
    import bb.msg
  File "/srv/pokybuild/yocto-worker/genericarm64/build/repos/bitbake/lib/bb/__init__.py", line 155, in <module>
    from bb import fetch2 as fetch
  File "/srv/pokybuild/yocto-worker/genericarm64/build/repos/bitbake/lib/bb/fetch2/__init__.py", line 2107, in <module>
    from . import curl
  File "/srv/pokybuild/yocto-worker/genericarm64/build/repos/bitbake/lib/bb/fetch2/curl.py", line 53, in <module>
    class Curl(Wget):
  File "/srv/pokybuild/yocto-worker/genericarm64/build/repos/bitbake/lib/bb/fetch2/curl.py", line 104, in Curl
    def _runcurl(self, ud: FetchData, d: DataSmart, command: str, quiet: bool, workdir: str | None = None):
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/3308
https://autobuilder.yoctoproject.org/valkyrie/#/builders/93/builds/3302

[1]: https://peps.python.org/pep-0604/
[2]: https://git.openembedded.org/bitbake/tree/lib/bb/__init__.py

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH 2/4] fetch2: make curl method activable with BB_FETCH_METHOD_HTTP

[Pascal Eberhard via B4 Relay]

From: Pascal Eberhard <pascal.eberhard@se.com>

wget is the default fetch method. curl fetch method can be activated by
setting bitbake variable:
  BB_FETCH_METHOD_HTTP = "curl"

Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
---
 lib/bb/fetch2/__init__.py |  2 ++
 lib/bb/fetch2/wget.py     | 10 ++++++++++
 2 files changed, 12 insertions(+)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index aaefd8602..dc1158b38 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -2107,6 +2107,7 @@ from . import gitannex
 from . import local
 from . import svn
 from . import wget
+from . import curl
 from . import ssh
 from . import sftp
 from . import s3
@@ -2123,6 +2124,7 @@ from . import gomod
 
 methods.append(local.Local())
 methods.append(wget.Wget())
+methods.append(curl.Curl())
 methods.append(svn.Svn())
 methods.append(git.Git())
 methods.append(gitsm.GitSM())
diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch2/wget.py
index 4e3505599..3b1993f29 100644
--- a/lib/bb/fetch2/wget.py
+++ b/lib/bb/fetch2/wget.py
@@ -60,10 +60,20 @@ class Wget(FetchMethod):
         """
         return (d.getVar("BB_CHECK_SSL_CERTS") or "1") != "0"
 
+    def is_enabled(self, d):
+        """
+        wget method is enabled when BB_FETCH_METHOD_HTTP = "wget" or by default
+        when BB_FETCH_METHOD_HTTP variable is not set.
+        """
+        method_http: str = d.getVar("BB_FETCH_METHOD_HTTP") or "wget"
+        return method_http == "wget"
+
     def supports(self, ud, d):
         """
         Check to see if a given url can be fetched with wget.
         """
+        if not self.is_enabled(d):
+            return False
         return ud.type in ['http', 'https', 'ftp', 'ftps']
 
     def recommends_checksum(self, urldata):

-- 
2.43.0

[PATCH 3/4] lib/tests/fetch2: add tests for curl method

[Pascal Eberhard via B4 Relay]

From: Pascal Eberhard <pascal.eberhard@se.com>

Duplicate wget fetch method test cases to test curl fetch method.

Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
---
 lib/bb/tests/fetch.py | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 74eb73472..5c587d7b4 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -1550,6 +1550,7 @@ class FetchLatestVersionTest(FetcherTest):
 
                     url = "http://127.0.0.1:%s/%s" % (port, data.path)
                     ud = bb.fetch2.FetchData(url, self.d)
+                    self.assertEqual(ud.method.__class__.__name__, "Wget")
                     pupver = ud.method.latest_versionstring(ud, self.d)
                     verstring = pupver[0]
                     self.assertTrue(verstring, msg="Could not find upstream version for %s" % data.pn)
@@ -1558,6 +1559,35 @@ class FetchLatestVersionTest(FetcherTest):
         finally:
             server.stop()
 
+    def test_curl_latest_versionstring(self):
+        self.d.setVar("BB_FETCH_METHOD_HTTP", "curl")
+        testdata = os.path.dirname(os.path.abspath(__file__)) + "/fetch-testdata"
+        server = HTTPService(testdata, host="127.0.0.1")
+        server.start()
+        port = server.port
+        try:
+            for data, v in self.test_wget_uris.items():
+                with self.subTest(pn=data.pn):
+                    self.d.setVar("PN", data.pn)
+                    self.d.setVar("PV", data.pv)
+                    if data.check_uri:
+                        checkuri = "http://127.0.0.1:%s/%s" % (port, data.check_uri)
+                        self.d.setVar("UPSTREAM_CHECK_URI", checkuri)
+                    if data.check_regex:
+                        self.d.setVar("UPSTREAM_CHECK_REGEX", data.check_regex)
+
+                    url = "http://127.0.0.1:%s/%s" % (port, data.path)
+                    ud = bb.fetch2.FetchData(url, self.d)
+                    self.assertEqual(ud.method.__class__.__name__, "Curl")
+                    pupver = ud.method.latest_versionstring(ud, self.d)
+                    verstring = pupver[0]
+                    self.assertTrue(verstring, msg="Could not find upstream version for %s" % data.pn)
+                    r = bb.utils.vercmp_string(v, verstring)
+                    self.assertTrue(r == -1 or r == 0, msg="Package %s, version: %s <= %s" % (data.pn, v, verstring))
+        finally:
+            server.stop()
+        self.d.delVar("BB_FETCH_METHOD_HTTP")
+
     @skipIfNoNetwork()
     def test_crate_latest_versionstring(self):
         for k, v in self.test_crate_uris.items():
@@ -1590,6 +1620,7 @@ class FetchCheckStatusTest(FetcherTest):
             with self.subTest(url=u):
                 ud = fetch.ud[u]
                 m = ud.method
+                self.assertEqual(ud.method.__class__.__name__, "Wget")
                 ret = m.checkstatus(fetch, ud, self.d)
                 self.assertTrue(ret, msg="URI %s, can't check status" % (u))
 
@@ -1605,10 +1636,47 @@ class FetchCheckStatusTest(FetcherTest):
             with self.subTest(url=u):
                 ud = fetch.ud[u]
                 m = ud.method
+                self.assertEqual(ud.method.__class__.__name__, "Wget")
+                ret = m.checkstatus(fetch, ud, self.d)
+                self.assertTrue(ret, msg="URI %s, can't check status" % (u))
+
+        connection_cache.close_connections()
+
+    @skipIfNoNetwork()
+    def test_curl_checkstatus(self):
+        self.d.setVar("BB_FETCH_METHOD_HTTP", "curl")
+
+        fetch = bb.fetch2.Fetch(self.test_wget_uris, self.d)
+        for u in self.test_wget_uris:
+            with self.subTest(url=u):
+                ud = fetch.ud[u]
+                m = ud.method
+                self.assertEqual(ud.method.__class__.__name__, "Curl")
+                ret = m.checkstatus(fetch, ud, self.d)
+                self.assertTrue(ret, msg="URI %s, can't check status" % (u))
+
+        self.d.delVar("BB_FETCH_METHOD_HTTP")
+
+    @skipIfNoNetwork()
+    def test_curl_checkstatus_connection_cache(self):
+        from bb.fetch2 import FetchConnectionCache
+
+        self.d.setVar("BB_FETCH_METHOD_HTTP", "curl")
+
+        connection_cache = FetchConnectionCache()
+        fetch = bb.fetch2.Fetch(self.test_wget_uris, self.d,
+                    connection_cache = connection_cache)
+
+        for u in self.test_wget_uris:
+            with self.subTest(url=u):
+                ud = fetch.ud[u]
+                m = ud.method
+                self.assertEqual(ud.method.__class__.__name__, "Curl")
                 ret = m.checkstatus(fetch, ud, self.d)
                 self.assertTrue(ret, msg="URI %s, can't check status" % (u))
 
         connection_cache.close_connections()
+        self.d.delVar("BB_FETCH_METHOD_HTTP")
 
 
 class GitMakeShallowTest(FetcherTest):

-- 
2.43.0

[PATCH 4/4] doc: bitbake-user-manual-ref-variables: describe BB_FETCH_METHOD_HTTP variable

[Pascal Eberhard via B4 Relay]

From: Pascal Eberhard <pascal.eberhard@se.com>

add BB_FETCH_METHOD_HTTP variable to select between wget and curl method
for http, https, ftp, ftps protocols.

Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
---
 doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
index 06bd53619..c459ed3af 100644
--- a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
+++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -311,6 +311,11 @@ overview of their function and contents.
       search the main :term:`SRC_URI` or
       :term:`MIRRORS`.
 
+   :term:`BB_FETCH_METHOD_HTTP`
+      When set to "curl", cause BitBake's fetcher to use curl method for
+      downloading web artifacts via http, https, ftp or ftps.
+      When not set (default) or set to "wget", wget method is used.
+
    :term:`BB_FILENAME`
       Contains the filename of the recipe that owns the currently running
       task. For example, if the ``do_fetch`` task that resides in the

-- 
2.43.0

Re: [bitbake-devel] [PATCH 0/4] fetch2: add alternative fetch method based on curl

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 1777 bytes --]

On Thu, 2026-03-05 at 16:32 +0100, Pascal Eberhard via B4 Relay via
lists.openembedded.org wrote:
> wget fetch method is used for web downloads with http, https, ftp, ftps
> protocols. wget cmdline tool is missing some features such as hostname
> resolution by the proxy when using SOCKS5 proxy. SSH tunnel provides
> this feature for example.
> 
> This curl fetch method is based on curl cmdline tool and provides the
> same protocols as wget and supports SOCKS5 hostname resolution.
> This class inherits wget method in order to avoid code duplication as
> much as possible.
> 
> wget remains the default download method. curl fetch method can be
> enabled by setting a new bitbake variable:
>   BB_FETCH_METHOD_HTTP = "curl"
> 
> The hostname resolution by SOCKS5 proxy is activated by setting
> environment variable:
>   all_proxy="socks5h://...""
> 
> a patch on ce-core will be sent if this proposal is accepted to
> conditionnaly add curl to HOSTTOOLS variable in meta/conf/bitbake.conf:
> HOSTTOOLS += "${@bb.utils.contains('BB_FETCH_METHOD_HTTP', 'curl', 'curl','', d)}"
> 
> Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>

Hi Pascal,

Adding a second HTTP fetcher to bitbake increases complexity and
maintenance cost. The behaviour of wget and curl is unlikely to match
exactly and so issues will be caused when people switch from one to the
other (for whatever reason).

Is it just socks5h:// support that you need? Do you know if there is a
bug/feature request open with wget upstream regarding socks5h support?

Also, do you know the minimum curl version required to support socks5h?
This may be important as we support multiple distros which may have
different curl versions.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

RE: [bitbake-devel] [PATCH 0/4] fetch2: add alternative fetch method based on curl

[Pascal EBERHARD]

On Thu Mar 05, 2026 at 19:09, Paul Barker wrote:
> On Thu, 2026-03-05 at 16:32 +0100, Pascal Eberhard via B4 Relay via
> lists.openembedded.org wrote:
> 
> > wget fetch method is used for web downloads with http, https, ftp, ftps
> > protocols. wget cmdline tool is missing some features such as hostname
> > resolution by the proxy when using SOCKS5 proxy. SSH tunnel provides
> > this feature for example.
> >
> > This curl fetch method is based on curl cmdline tool and provides the
> > same protocols as wget and supports SOCKS5 hostname resolution.
> > This class inherits wget method in order to avoid code duplication as
> > much as possible.
> >
> > wget remains the default download method. curl fetch method can be
> > enabled by setting a new bitbake variable:
> >   BB_FETCH_METHOD_HTTP = "curl"
> >
> > The hostname resolution by SOCKS5 proxy is activated by setting
> > environment variable:
> >   all_proxy="socks5h://...""
> >
> > a patch on ce-core will be sent if this proposal is accepted to
> > conditionnaly add curl to HOSTTOOLS variable in meta/conf/bitbake.conf:
> > HOSTTOOLS += "${@bb.utils.contains('BB_FETCH_METHOD_HTTP', 'curl', 'curl','', d)}"
> >
> > Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com>
> 
> 
> Hi Pascal,

Hi Paul,

> Adding a second HTTP fetcher to bitbake increases complexity and
> maintenance cost. The behaviour of wget and curl is unlikely to match
> exactly and so issues will be caused when people switch from one to the
> other (for whatever reason).
 
Indeed, I understand having to ponder the value given the increase in
the maintenance burden.

> Is it just socks5h:// support that you need? Do you know if there is a
> bug/feature request open with wget upstream regarding socks5h support?

Actually, wget has no support at all for SOCKS proxy :(. A workaround
with tsocks works for SOCKS4 and SOCKS5, but hostname resolution isn’t
possible that way. So implementing socks5h in wget would require
implementing SOCKS proxy support from scratch, which explain the choice
of curl.

> Also, do you know the minimum curl version required to support socks5h?
> This may be important as we support multiple distros which may have
> different curl versions.

The 'socks5h' scheme has been supported since 2011. The motivation for
this proposal was SOCKS proxy support, but curl may have other benefits
than I am not aware of.


Best regards,
Pascal