[Buildroot] github tarball urls: http vs https

[Buildroot] github tarball urls: http vs https

[Thomas De Schampheleire]

Hi,

Packages that are hosted on github and downloaded with the tarball
method, can either have a http or https URL. It seems that a download
from http is redirected to the corresponding https URL. To avoid such
an unnecessary redirect, we could update all github .mk files to use
https directly.

I vaguely recall a discussion on the mailing list about this, but I
don't know what the outcome was. Was there a problem using the https
URLs with respect to certificates?

If we can clarify this point, we can cleanup and update the manual
(which talks about http).

Thanks,
Thomas

[Buildroot] github tarball urls: http vs https

[Jerzy Grzegorek]

Hi Thomas,


> Hi,
>
> Packages that are hosted on github and downloaded with the tarball
> method, can either have a http or https URL. It seems that a download
> from http is redirected to the corresponding https URL. To avoid such
> an unnecessary redirect, we could update all github .mk files to use
> https directly.
>
> I vaguely recall a discussion on the mailing list about this, but I
> don't know what the outcome was. Was there a problem using the https
> URLs with respect to certificates?

It was my proposal.
Please look here:
http://lists.busybox.net/pipermail/buildroot/2013-October/079209.html

Regards,
Jerzy


> If we can clarify this point, we can cleanup and update the manual
> (which talks about http).
>
> Thanks,
> Thomas
>

[Buildroot] github tarball urls: http vs https

[Thomas De Schampheleire]

Hi Jerzy, Arnout, all,

On Sat, Nov 2, 2013 at 6:47 PM, Jerzy Grzegorek
<jerzy.grzegorek@trzebnica.net> wrote:
[..]
>> Packages that are hosted on github and downloaded with the tarball
>> method, can either have a http or https URL. It seems that a download
>> from http is redirected to the corresponding https URL. To avoid such
>> an unnecessary redirect, we could update all github .mk files to use
>> https directly.
>>
>> I vaguely recall a discussion on the mailing list about this, but I
>> don't know what the outcome was. Was there a problem using the https
>> URLs with respect to certificates?
>
>
> It was my proposal.
> Please look here:
> http://lists.busybox.net/pipermail/buildroot/2013-October/079209.html
>

Thanks for the link. However, besides a comment from Arnout, the
discussion was more about the VERSION part rather than the URL itself.

Arnout, in that thread you wrote:
"Also you change the URL to https here. With the recent problems with
https URLs that we've seen on the autobuilders recently, I wonder if this
is a good idea?"

Could you clarify what problems you were talking about?

Thanks,
Thomas

[Buildroot] github tarball urls: http vs https

[Arnout Vandecappelle]

On 02/11/13 19:04, Thomas De Schampheleire wrote:
> Hi Jerzy, Arnout, all,
>
> On Sat, Nov 2, 2013 at 6:47 PM, Jerzy Grzegorek
> <jerzy.grzegorek@trzebnica.net> wrote:
> [..]
>>> Packages that are hosted on github and downloaded with the tarball
>>> method, can either have a http or https URL. It seems that a download
>>> from http is redirected to the corresponding https URL. To avoid such
>>> an unnecessary redirect, we could update all github .mk files to use
>>> https directly.
>>>
>>> I vaguely recall a discussion on the mailing list about this, but I
>>> don't know what the outcome was. Was there a problem using the https
>>> URLs with respect to certificates?
>>
>>
>> It was my proposal.
>> Please look here:
>> http://lists.busybox.net/pipermail/buildroot/2013-October/079209.html
>>
>
> Thanks for the link. However, besides a comment from Arnout, the
> discussion was more about the VERSION part rather than the URL itself.
>
> Arnout, in that thread you wrote:
> "Also you change the URL to https here. With the recent problems with
> https URLs that we've seen on the autobuilders recently, I wonder if this
> is a good idea?"

  First of all: I didn't realize that the http URL just redirects to an 
https URL. In that case, obviously, using the https URL is better.

>
> Could you clarify what problems you were talking about?

  IIRC, at some point there was a problem that a download site used a 
certificate signed by a recent CA that was not included in the 
autobuilder's trusted certificate list, so wget would not accept it. It 
was discussed that an option was to run wget with --no-check-certificate, 
but this would defeat the purpose of https so was rejected. Of course, 
using an http URL instead of an https has the same result.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] github tarball urls: http vs https

[Thomas De Schampheleire]

Hi Arnout,

On Mon, Nov 4, 2013 at 7:47 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 02/11/13 19:04, Thomas De Schampheleire wrote:
[..]
>> Arnout, in that thread you wrote:
>> "Also you change the URL to https here. With the recent problems with
>> https URLs that we've seen on the autobuilders recently, I wonder if this
>> is a good idea?"
>
>
>  First of all: I didn't realize that the http URL just redirects to an https
> URL. In that case, obviously, using the https URL is better.
>
>
>>
>> Could you clarify what problems you were talking about?
>
>
>  IIRC, at some point there was a problem that a download site used a
> certificate signed by a recent CA that was not included in the autobuilder's
> trusted certificate list, so wget would not accept it. It was discussed that
> an option was to run wget with --no-check-certificate, but this would defeat
> the purpose of https so was rejected. Of course, using an http URL instead
> of an https has the same result.

But this seems to be a temporary problem only.
Besides, what happens in that scenario if you try http, and the server
redirects it to https? I would expect the certificate to fail, or does
wget pass an implicit --no-check-certificate in this case?

Best regards,
Thomas

[Buildroot] github tarball urls: http vs https

[Arnout Vandecappelle]

On 04/11/13 09:33, Thomas De Schampheleire wrote:
> Hi Arnout,
>
> On Mon, Nov 4, 2013 at 7:47 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
>> On 02/11/13 19:04, Thomas De Schampheleire wrote:
> [..]
>>> Arnout, in that thread you wrote:
>>> "Also you change the URL to https here. With the recent problems with
>>> https URLs that we've seen on the autobuilders recently, I wonder if this
>>> is a good idea?"
>>
>>
>>   First of all: I didn't realize that the http URL just redirects to an https
>> URL. In that case, obviously, using the https URL is better.
>>
>>
>>>
>>> Could you clarify what problems you were talking about?
>>
>>
>>   IIRC, at some point there was a problem that a download site used a
>> certificate signed by a recent CA that was not included in the autobuilder's
>> trusted certificate list, so wget would not accept it. It was discussed that
>> an option was to run wget with --no-check-certificate, but this would defeat
>> the purpose of https so was rejected. Of course, using an http URL instead
>> of an https has the same result.
>
> But this seems to be a temporary problem only.

  "Temporary" until the autobuilder's CA certificates are updated, you mean?

> Besides, what happens in that scenario if you try http, and the server
> redirects it to https? I would expect the certificate to fail, or does
> wget pass an implicit --no-check-certificate in this case?

  If it redirects, it will still fail. That's why using the https URL is 
better in that case, as I mentioned above.

  Regards,
  Arnout

>
> Best regards,
> Thomas
>
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] github tarball urls: http vs https

[Thomas De Schampheleire]

Hi Arnout, all,

On Mon, Nov 4, 2013 at 10:28 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 04/11/13 09:33, Thomas De Schampheleire wrote:
>>

>>>   IIRC, at some point there was a problem that a download site used a
>>> certificate signed by a recent CA that was not included in the
>>> autobuilder's
>>> trusted certificate list, so wget would not accept it. It was discussed
>>> that
>>> an option was to run wget with --no-check-certificate, but this would
>>> defeat
>>> the purpose of https so was rejected. Of course, using an http URL
>>> instead
>>> of an https has the same result.
>>
>>
>> But this seems to be a temporary problem only.
>
>
>  "Temporary" until the autobuilder's CA certificates are updated, you mean?
>

Yes, that's what I meant.

>
>> Besides, what happens in that scenario if you try http, and the server
>> redirects it to https? I would expect the certificate to fail, or does
>> wget pass an implicit --no-check-certificate in this case?
>
>
>  If it redirects, it will still fail. That's why using the https URL is
> better in that case, as I mentioned above.
>

Ok, great, so it looks like we have a plan: we implement a github
helper, and have that use https directly.

Now we need a volunteer to do this. I have no problem in doing it but
it would end up on my todo list first, so if there is anyone else who
has time, then please be my guest.

Best regards,
Thomas