Re: [meta-raspberrypi][PATCH] firmware.inc: fetch from SVN instead of Git

[Andrei Gherzan <andrei@gherzan.ro>]

On Mon, Aug 10, 2015 at 01:03:57AM -0700, Khem Raj wrote:
> On Sun, Aug 9, 2015 at 4:57 PM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> > Hi,
> >
> > On Mon, Jul 20, 2015 at 09:59:30AM -0600, Gary Thomas wrote:
> >> On 2015-07-19 15:34, Andrei Gherzan wrote:
> >> >Hello,
> >> >
> >> >--
> >> >Andrei Gherzan
> >> >
> >> >On Thu, Jul 16, 2015 at 7:53 PM, Jon Szymaniak <jon.szymaniak@gmail.com <mailto:jon.szymaniak@gmail.com>> wrote:
> >> >
> >> >    Hi all,
> >> >
> >> >        >
> >> >        > So there is no support for depth clones until 2.5.0? I didn't really
> >> >        > understand.
> >> >
> >> >        Well, shallow clones are supported but only for branch tips, which is
> >> >        not what we need. This feature for shallow cloning a specific revision
> >> >        is available only in git 2.5.0+. Also, this feature needs a server-side
> >> >        configuration option to be enabled, in order to work properly.
> >> >
> >> >        Regards,
> >> >        Nikolay
> >> >
> >> >
> >> >    I'd argue that this whole issue with the whole meta-raspberrypi firmware.inc download is more than just slow, inconvenient download.  I've left builds running all night (8+
> >> >    hours on a 30Mib/s residential link) that just hang on this, usually timing out.  I initially thought it was just me, but am hearing others confirm this as well.
> >> >
> >> >    As such, I just wanted to continue this conversation.  It sounds like git fetch's --depth is the best option on the table, but has the issues Nikolay has described.  What are
> >> >    your thoughts on the following?
> >> >
> >> >    (1) We create a git-native and build a version that supports this the fetch depth?
> >> >
> >> >    I suspect this could be made to work, but haven't dug into what dependencies git may have and how that would play out on various LTS distros. My knee-jerk is that has too high
> >> >    of a risk/benefit ration, given that we're talking about 1 repo.
> >> >
> >> >    (2) We update the git fetcher to check the git version and support a depth= option if the git version is sufficient.  If it is not, we spit out a warning and fall back to the
> >> >    current behavior.
> >> >
> >> >    Neither (1) nor (2) address Nikolay's point that --depth requires server-side support.  However, I'd argue this is something you'd be testing and verifying when writing the
> >> >    recipe.  Is this a reasonable assertion? How likely is it that a server supporting this would suddenly be re-configured?
> >> >
> >> >    (3) We request that the upstream maintainer of meta-raspberrypi use the GitHub Release feature [a] to post a tarball of a known checksum at somewhat regular intervals.  I'm
> >> >    told by a few package maintainers that while the tarballs that it generates for specific changesets are subject to change, that the tarballs it autogenerates when using its
> >> >    Releases feature do not.  However, I have not confirmed this.  If this is false, then one can upload a tarball with known checksums to the release as an attachment; I would be
> >> >    *shocked* if they touch your attachments.
> >> >
> >> >    While (3) is a nice "not our problem" solution, I think (2) might have some benefits for other recipes later.  Any other ideas?
> >> >
> >> >
> >> >Definitely 2 is the best opinion in my opinion too. I'm wondering if there is any work started in this direction.
> >> >
> >>
> >> It would sure be nice to get this fixed!  The latest download
> >> (I always save the tarballs for the next time) is over 4GB!!
> >>
> >
> > So what is the conlcusion on this? Should we switch on SVN temporary?
>
> Step back a bit and look at what this firmware repo is all about. Its
> bunch of binaries in there. Git is a bad choice for such a repo. Since
> binary blobs whenever updates will add that much of data to size of
> metadata and soon it will go into tens of gigabytes so its a building
> avalanche. Ideally this firmware should have been released as tarballs
> in first place. Can you work with owners who are releasing this in
> this form to reconsider releasing it in discrete tarballs ?

This is not under our control. I will try to talk with rpi people:
https://github.com/raspberrypi/firmware/issues/455

Khem, could you add a comment there too?

Thanks,

--
Andrei Gherzan