* [Buildroot] Download tag as archive from gitlab @ 2017-03-06 23:28 Angelo Compagnucci 2017-03-07 7:11 ` Yann E. MORIN 0 siblings, 1 reply; 6+ messages in thread From: Angelo Compagnucci @ 2017-03-06 23:28 UTC (permalink / raw) To: buildroot Hello guys, I'm hitting some bumps trying to download a tag from gitlab as an archive, the url format is like this: https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 I can succesfully download the file, but then it's named "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks an extensions and buildroot cannot choose the right helper to decompress it. How such an url can be downloaded in buildroot? Thanks! -- Profile: http://it.linkedin.com/in/compagnucciangelo ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] Download tag as archive from gitlab 2017-03-06 23:28 [Buildroot] Download tag as archive from gitlab Angelo Compagnucci @ 2017-03-07 7:11 ` Yann E. MORIN 2017-03-07 8:36 ` Arnout Vandecappelle 0 siblings, 1 reply; 6+ messages in thread From: Yann E. MORIN @ 2017-03-07 7:11 UTC (permalink / raw) To: buildroot Angelo, All, On 2017-03-07 00:28 +0100, Angelo Compagnucci spake thusly: > I'm hitting some bumps trying to download a tag from gitlab as an > archive, the url format is like this: > > https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 > > I can succesfully download the file, but then it's named > "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks > an extensions and buildroot cannot choose the right helper to > decompress it. > > How such an url can be downloaded in buildroot? The only solution is to download it with a git clone: FOO_VERSION = TAG_OR_SHA1 FOO_SITE = https://gitlab.com/gitlab-org/gitlab-ce FOO_SITE_METHOD = git Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] Download tag as archive from gitlab 2017-03-07 7:11 ` Yann E. MORIN @ 2017-03-07 8:36 ` Arnout Vandecappelle 2017-03-07 8:40 ` Angelo Compagnucci 2017-03-07 8:45 ` Yann E. MORIN 0 siblings, 2 replies; 6+ messages in thread From: Arnout Vandecappelle @ 2017-03-07 8:36 UTC (permalink / raw) To: buildroot On 07-03-17 08:11, Yann E. MORIN wrote: > Angelo, All, > > On 2017-03-07 00:28 +0100, Angelo Compagnucci spake thusly: >> I'm hitting some bumps trying to download a tag from gitlab as an >> archive, the url format is like this: >> >> https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 >> >> I can succesfully download the file, but then it's named >> "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks >> an extensions and buildroot cannot choose the right helper to >> decompress it. >> >> How such an url can be downloaded in buildroot? > > The only solution is to download it with a git clone: The only solution you can think of :-) You can use the following hack: FOO_VERSION = v6.9.2 FOO_SITE = https://.../archive.tar.bz2?ref=$(FOO_VERSION)&filename= FOO_SOURCE = foo-$(FOO_VERSION).tar.bz2 This will lead to a wget of https://.../archive.tar.bz2?ref=v6.9.2&filename=/foo-v6.9.2.tar.bz2 to the file foo-v6.9.2.tar.bz2. It works because in a GET request, any unknown variables are typically ignored. Last time I tried gitlab did ignore them. Of course you can use anything you like for FOO_SOURCE, but it's convenient to use the canonical name. Regards, Arnout > > FOO_VERSION = TAG_OR_SHA1 > FOO_SITE = https://gitlab.com/gitlab-org/gitlab-ce > FOO_SITE_METHOD = git > > Regards, > Yann E. MORIN. > -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] Download tag as archive from gitlab 2017-03-07 8:36 ` Arnout Vandecappelle @ 2017-03-07 8:40 ` Angelo Compagnucci 2017-03-07 8:45 ` Yann E. MORIN 1 sibling, 0 replies; 6+ messages in thread From: Angelo Compagnucci @ 2017-03-07 8:40 UTC (permalink / raw) To: buildroot Dear Arnout, Yann, 2017-03-07 9:36 GMT+01:00 Arnout Vandecappelle <arnout@mind.be>: > > > On 07-03-17 08:11, Yann E. MORIN wrote: >> Angelo, All, >> >> On 2017-03-07 00:28 +0100, Angelo Compagnucci spake thusly: >>> I'm hitting some bumps trying to download a tag from gitlab as an >>> archive, the url format is like this: >>> >>> https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 >>> >>> I can succesfully download the file, but then it's named >>> "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks >>> an extensions and buildroot cannot choose the right helper to >>> decompress it. >>> >>> How such an url can be downloaded in buildroot? >> >> The only solution is to download it with a git clone: > > The only solution you can think of :-) > > You can use the following hack: > > FOO_VERSION = v6.9.2 > FOO_SITE = https://.../archive.tar.bz2?ref=$(FOO_VERSION)&filename= > FOO_SOURCE = foo-$(FOO_VERSION).tar.bz2 > > This will lead to a wget of > https://.../archive.tar.bz2?ref=v6.9.2&filename=/foo-v6.9.2.tar.bz2 > to the file foo-v6.9.2.tar.bz2. > > It works because in a GET request, any unknown variables are typically ignored. > Last time I tried gitlab did ignore them. > > Of course you can use anything you like for FOO_SOURCE, but it's convenient to > use the canonical name. I was arrived at the same solution just minutes ago! I think just adding a gitlab helper like the github one with the hack included! Thanks! Sincerely, Angelo > > Regards, > Arnout > > >> >> FOO_VERSION = TAG_OR_SHA1 >> FOO_SITE = https://gitlab.com/gitlab-org/gitlab-ce >> FOO_SITE_METHOD = git >> >> Regards, >> Yann E. MORIN. >> > > -- > Arnout Vandecappelle arnout at mind be > Senior Embedded Software Architect +32-16-286500 > Essensium/Mind http://www.mind.be > G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven > LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle > GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF -- Profile: http://it.linkedin.com/in/compagnucciangelo ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] Download tag as archive from gitlab 2017-03-07 8:36 ` Arnout Vandecappelle 2017-03-07 8:40 ` Angelo Compagnucci @ 2017-03-07 8:45 ` Yann E. MORIN 2017-03-07 9:18 ` Angelo Compagnucci 1 sibling, 1 reply; 6+ messages in thread From: Yann E. MORIN @ 2017-03-07 8:45 UTC (permalink / raw) To: buildroot Arnout, All, On 2017-03-07 09:36 +0100, Arnout Vandecappelle spake thusly: > On 07-03-17 08:11, Yann E. MORIN wrote: > > Angelo, All, > > > > On 2017-03-07 00:28 +0100, Angelo Compagnucci spake thusly: > >> I'm hitting some bumps trying to download a tag from gitlab as an > >> archive, the url format is like this: > >> > >> https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 > >> > >> I can succesfully download the file, but then it's named > >> "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks > >> an extensions and buildroot cannot choose the right helper to > >> decompress it. > >> > >> How such an url can be downloaded in buildroot? > > > > The only solution is to download it with a git clone: > > The only solution you can think of :-) > > You can use the following hack: > > FOO_VERSION = v6.9.2 > FOO_SITE = https://.../archive.tar.bz2?ref=$(FOO_VERSION)&filename= > FOO_SOURCE = foo-$(FOO_VERSION).tar.bz2 Yeah, I know that trick. I think you even mentioned it not so long ago for that very same issue. Although this is a neat trick, it still is a trick, it's not obvious, it's not documented, it's ugly. ;-) So I think we should refrain from accepting such tricks. If upstream does not have a sane archive download solution, then I prefer that we do a git clone instead; it's much obvious what is going on. Yes, this usually means bigger/slower downloads, but then people should go complain to the culprit (here, gitlab). Or we add a gitlab helper. Meh... No. ;-) Regards, Yann E. MORIN. > This will lead to a wget of > https://.../archive.tar.bz2?ref=v6.9.2&filename=/foo-v6.9.2.tar.bz2 > to the file foo-v6.9.2.tar.bz2. > > It works because in a GET request, any unknown variables are typically ignored. > Last time I tried gitlab did ignore them. > > Of course you can use anything you like for FOO_SOURCE, but it's convenient to > use the canonical name. > > Regards, > Arnout > > > > > > FOO_VERSION = TAG_OR_SHA1 > > FOO_SITE = https://gitlab.com/gitlab-org/gitlab-ce > > FOO_SITE_METHOD = git > > > > Regards, > > Yann E. MORIN. > > > > -- > Arnout Vandecappelle arnout at mind be > Senior Embedded Software Architect +32-16-286500 > Essensium/Mind http://www.mind.be > G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven > LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle > GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] Download tag as archive from gitlab 2017-03-07 8:45 ` Yann E. MORIN @ 2017-03-07 9:18 ` Angelo Compagnucci 0 siblings, 0 replies; 6+ messages in thread From: Angelo Compagnucci @ 2017-03-07 9:18 UTC (permalink / raw) To: buildroot Yann, All 2017-03-07 9:45 GMT+01:00 Yann E. MORIN <yann.morin.1998@free.fr>: > Arnout, All, > > On 2017-03-07 09:36 +0100, Arnout Vandecappelle spake thusly: >> On 07-03-17 08:11, Yann E. MORIN wrote: >> > Angelo, All, >> > >> > On 2017-03-07 00:28 +0100, Angelo Compagnucci spake thusly: >> >> I'm hitting some bumps trying to download a tag from gitlab as an >> >> archive, the url format is like this: >> >> >> >> https://gitlab.com/gitlab-org/gitlab-ce/repository/archive.tar.bz2?ref=v6.9.2 >> >> >> >> I can succesfully download the file, but then it's named >> >> "archive.tar.bz2?ref=v6.9.2" but it cannot be extracted cause it lacks >> >> an extensions and buildroot cannot choose the right helper to >> >> decompress it. >> >> >> >> How such an url can be downloaded in buildroot? >> > >> > The only solution is to download it with a git clone: >> >> The only solution you can think of :-) >> >> You can use the following hack: >> >> FOO_VERSION = v6.9.2 >> FOO_SITE = https://.../archive.tar.bz2?ref=$(FOO_VERSION)&filename= >> FOO_SOURCE = foo-$(FOO_VERSION).tar.bz2 > > Yeah, I know that trick. I think you even mentioned it not so long ago > for that very same issue. > > Although this is a neat trick, it still is a trick, it's not obvious, > it's not documented, it's ugly. ;-) > > So I think we should refrain from accepting such tricks. > > If upstream does not have a sane archive download solution, then I > prefer that we do a git clone instead; it's much obvious what is going > on. Yes, this usually means bigger/slower downloads, but then people > should go complain to the culprit (here, gitlab). > > Or we add a gitlab helper. Meh... No. ;-) It's not entirely true: gitlab returns Content-Disposition=attachment; filename="gitlab-ce-v6.9.2-e46b644a8857a53ed3f6c3f64b224bb74b06fd8e.tar.gz" In the header for the GET request, so it behaves correctly. Indeed if you download the file with a browser you will obtain a sensible name. The filename is in the form <project>-<version>-<sha>.ext . Unfortunately, the buildroot download helper DOWNLOAD_WGET is a bit rigid: it overwrites the filename and requires the url to be in a certain form. Content-Disposition is heavily used in modern web services where the filename on the cloud usually has a form of a pointer to an object and the real filename it's stored somewhere and retrieved via an API. wget can use for example --content-disposition to get the filename. This also makes the progress bar behave correctly cause it inspects the header for the file size. Sincerely, Angelo > > Regards, > Yann E. MORIN. > >> This will lead to a wget of >> https://.../archive.tar.bz2?ref=v6.9.2&filename=/foo-v6.9.2.tar.bz2 >> to the file foo-v6.9.2.tar.bz2. >> >> It works because in a GET request, any unknown variables are typically ignored. >> Last time I tried gitlab did ignore them. >> >> Of course you can use anything you like for FOO_SOURCE, but it's convenient to >> use the canonical name. >> >> Regards, >> Arnout >> >> >> > >> > FOO_VERSION = TAG_OR_SHA1 >> > FOO_SITE = https://gitlab.com/gitlab-org/gitlab-ce >> > FOO_SITE_METHOD = git >> > >> > Regards, >> > Yann E. MORIN. >> > >> >> -- >> Arnout Vandecappelle arnout at mind be >> Senior Embedded Software Architect +32-16-286500 >> Essensium/Mind http://www.mind.be >> G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven >> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle >> GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF > > -- > .-----------------.--------------------.------------------.--------------------. > | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | > | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | > | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | > | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | > '------------------------------^-------^------------------^--------------------' -- Profile: http://it.linkedin.com/in/compagnucciangelo ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-03-07 9:18 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-03-06 23:28 [Buildroot] Download tag as archive from gitlab Angelo Compagnucci 2017-03-07 7:11 ` Yann E. MORIN 2017-03-07 8:36 ` Arnout Vandecappelle 2017-03-07 8:40 ` Angelo Compagnucci 2017-03-07 8:45 ` Yann E. MORIN 2017-03-07 9:18 ` Angelo Compagnucci
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox