Re: [PATCH v4] fetch2/wget: include parameters in URL

["Patrick Williams" <patrick@stwcx.xyz>]

[-- Attachment #1: Type: text/plain, Size: 2407 bytes --]

On Wed, Dec 30, 2020 at 04:03:23PM -0600, Patrick Williams wrote:
> The wget fetcher currently silently drops all parameters and has
> been that way since it was originally introduced in fetch via 080eb8764.
> This prevents use in SOURCE_MIRROR_URL and SRC_URI which require
> additional parameters.
> 
> Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
> ---
>  lib/bb/fetch2/wget.py | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/bb/fetch2/wget.py b/lib/bb/fetch2/wget.py
> index e6d9f528..0019f10c 100644
> --- a/lib/bb/fetch2/wget.py
> +++ b/lib/bb/fetch2/wget.py
> @@ -91,7 +91,14 @@ class Wget(FetchMethod):
>  
>          fetchcmd = self.basecmd
>  
> -        if 'downloadfilename' in ud.parm:
> +        # Join the URL components together.
> +        url_parts = ud.url.split(";")
> +        if len(url_parts) == 1:
> +            uri = url_parts[0]
> +        else:
> +            uri = url_parts[0] + "?" + "&".join(url_parts[1:])
> +
> +        if 'downloadfilename' in ud.parm or len(url_parts) != 1:
>              localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile)
>              bb.utils.mkdirhier(os.path.dirname(localpath))
>              fetchcmd += " -O %s" % shlex.quote(localpath)
> @@ -99,7 +106,6 @@ class Wget(FetchMethod):
>          if ud.user and ud.pswd:
>              fetchcmd += " --user=%s --password=%s --auth-no-challenge" % (ud.user, ud.pswd)
>  
> -        uri = ud.url.split(";")[0]

I ran into one more edge case trying to re-download a file from my
internal mirror service: netbase_6.1~bpo10+1.tar.xz .  The URL
contains a URL-quoted version of the filename.  Switched the code
to reuse the 'downloadfilename' logic earlier in the function, which
is safer anyhow because it uses shlex, etc.

>          if os.path.exists(ud.localpath):
>              # file exists, but we didnt complete it.. trying again..
>              fetchcmd += d.expand(" -c -P ${DL_DIR} '%s'" % uri)
> @@ -441,7 +447,7 @@ class Wget(FetchMethod):
>                      valid = 1
>                  elif self._vercmp(version, newver) < 0:
>                      version = newver
> -                
> +
>          pupver = re.sub('_', '.', version[1])
>  
>          bb.debug(3, "*** %s -> UpstreamVersion = %s (CurrentVersion = %s)" %
> -- 
> 2.26.2
> 

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]