Re: [PATCH] wget.py: support for semi-colons in URL

[Peter Urbanec <openembedded-devel@urbanec.net>]

I have not examined the code, but could the URL use the standard URL 
encoding mechanism to represent ; as %3B ?

The idea being that the initial "Bitbake URL" is parsed / split using 
the original string with semicolons and the URL escaped semicolon is 
passed through to a lower layer that uses that part of the string as an 
"Internet URL"

Cheers,
	Peter


On 28/07/15 02:10, Bystricky, Juro wrote:
> I agree that a generic solution is preferable. I also I think it is only
> prudent to support all valid URLs.
>
> The situation is complicated by the fact that at present it may not be
> possible to determine how to interpret the ‘;’ in SRC_URI.
>
> It can be part of the URL or it can be a SRC_URI delimiter/separator.
>
> As I don’t suppose changing the delimiter character from semi-colon to
> something else (i.e. an unsafe URL character)
>
> is an option, some other changes to the SRC_URI syntax are needed, for
> example:
>
> 1.Some form of escape sequence (i.e. ‘;;’ means treat as ‘;’) in URL, i.e.
>
> SRC_URI =
> "http://abc123.com/git/?p=gcc/gcc.git;;a=snapshot;;h=a5dd47;downloadfilename=abc.tar"
>
> 2.Quoted string implementation, using one of the “unsafe” URL characters
> such as ‘`’, to delimit  the URL
>
> SRC_URI = "’
> http://abc123.com/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47`;downloadfilename=abc.tar"
>
> 3.Something else ?
>
> Thanks
>
> Juro
>
> *From:*kergoth@gmail.com [mailto:kergoth@gmail.com] *On Behalf Of
> *Christopher Larson
> *Sent:* Saturday, July 25, 2015 12:15 PM
> *To:* Bystricky, Juro
> *Cc:* bitbake-devel@lists.openembedded.org; Purdie, Richard
> *Subject:* Re: [bitbake-devel] [PATCH] wget.py: support for semi-colons
> in URL
>
> On Sat, Jul 25, 2015 at 11:32 AM, Juro Bystricky
> <juro.bystricky@intel.com <mailto:juro.bystricky@intel.com>> wrote:
>
>     Some URLs contain semi-colons. For example, some GIT repositories
>     support downloading snapshots using URL such as:
>
>     "http://abc123.com/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47"
>
>     Currently there is no way to construct SRC_URI with such URLs.
>     Bitbake uses semi-colons in SRC_URI as delimiters for various
>     parameters.
>
>     This patch allows using of semi-colons in URLs: ';;' (double semi-colon)
>     in SRC_URI is processed  as a single semi-colon that is a part of a
>     URL name.
>     For example, the above URL would become:
>
>     SRC_URI = "http://abc123.com/git/?p=gcc/gcc.git;;a=snapshot;;h=a5dd47"
>
>     Note that it is not possible to determine the name of the downloaded
>     file
>     from URL alone. The onus to specify the proper name is on the user,
>     via the parameter 'downloadfilename':
>
>     SRC_URI =
>     "http://abc123.com/git/?p=gcc/gcc.git;;a=snapshot;;h=a5dd47;downloadfilename=myfile.tar.gz"
>
>     Signed-off-by: Juro Bystricky <juro.bystricky@intel.com
>     <mailto:juro.bystricky@intel.com>>
>
>
> In my opinion, if we’re going to support this, we should really support
> it in a generic way, in urldecode/urlencode & the URL class, and if
> needed leverage that in wget.
> --
>
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics
>
>
>