[Buildroot] feature or pebkac

[Buildroot] feature or pebkac

[Arnout Vandecappelle]

On 08/29/13 20:17, rh wrote:
> I have found that changing the url for a custom kernel tar ball does not
> trigger a rebuild when running make.  I have removed linux-custom dir
> and then make fetches the new tar ball and rebuilds.

  That's right. Buildroot doesn't try to be perfect for rebuilds when the 
configuration changes - but if it is simple to fix, patches are welcome!


> I have found that when I specify a kernel git repo that I get the whole
> repo.

  If you specify a tag, then it should do a shallow clone.


> But I don't need all that and I now use --single-branch argument
> to git clone followed by git archive to create a tar ball.

  That's a great idea! However, this option is only available since git 
1.7.10 and I think that many production environments still have older git 
versions.

> I do this for repos
> that have no snapshot or tar ball. I thought that this feature might exist
> but I couldn't find it.  Maybe I can edit some config manually?

  There's no config for that, it's part of the core infrastructure.


  Regards,
  Arnout

>
> Would it be a useful feature for others if it's not there now?
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] feature or pebkac

[Thomas De Schampheleire]

Hi,

On Thu, Aug 29, 2013 at 9:05 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 08/29/13 20:17, rh wrote:
>>
>> I have found that changing the url for a custom kernel tar ball does not
>> trigger a rebuild when running make.  I have removed linux-custom dir
>> and then make fetches the new tar ball and rebuilds.
>
>
>  That's right. Buildroot doesn't try to be perfect for rebuilds when the
> configuration changes - but if it is simple to fix, patches are welcome!

I once proposed a patch for this particular case:
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-v2-linux-custom-tarball-automatically-detect-a-change-in-tarball-configuration-td11414.html
but unfortunately it never got accepted. One of the remarks was that
the build directory name is odd in the sense that it contains the name
of the tarball. But it fixes the problem perfectly, we are using it
all the time.

If there is interest in this, I can resubmit the patch.

Best regards,
Thomas

[Buildroot] feature or pebkac

[Thomas De Schampheleire]

On Fri, Aug 30, 2013 at 12:39 AM, rh <richard_hubbe11@lavabit.com> wrote:
> On Thu, 29 Aug 2013 22:25:55 +0200
> Thomas De Schampheleire <patrickdepinguin
> +buildroot at gmail.com> wrote:
>
>>
>> I once proposed a patch for this particular case:
>> http://buildroot-busybox.2317881.n4.nabble.com/PATCH-v2-linux-custom-tarball-automatically-detect-a-change-in-tarball-configuration-td11414.html
>> but unfortunately it never got accepted. One of the remarks was that
>> the build directory name is odd in the sense that it contains the name
>> of the tarball. But it fixes the problem perfectly, we are using it
>> all the time.
>>
>> If there is interest in this, I can resubmit the patch.
>
>
> Ok I read that patch.  Can we use sym links? If so I propose that
> we untar and use the tar ball name but then remove and create links
> pointing to the new directory. This way linux-custom is always there
> as a sym link and any dependencies on linux-custom are fulfilled.

Which dependencies on linux-custom are you talking about?
Inside buildroot, all references to the linux directory happen through
the variable LINUX_DIR, which will always be correct. Scripts outside
of buildroot may try to access the fixed 'linux-custom', but this
behavior is broken anyway because the board developer could choose to
switch to a vanilla kernel later on which would cause another
directory name too.

Thanks,
Thomas

[Buildroot] feature or pebkac

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Thu, 29 Aug 2013 21:05:53 +0200, Arnout Vandecappelle wrote:
> On 08/29/13 20:17, rh wrote:
> > I have found that changing the url for a custom kernel tar ball does not
> > trigger a rebuild when running make.  I have removed linux-custom dir
> > and then make fetches the new tar ball and rebuilds.
> 
>   That's right. Buildroot doesn't try to be perfect for rebuilds when the 
> configuration changes - but if it is simple to fix, patches are welcome!

As Thomas De Schampheleire later explained, the problem is that for
custom tarballs, we can't easily find a directory name that's unique
for each tarball, except by having a name that is a complete
translation of the tarball URL, such as:

output/build/linux-http-www.company.com-pub-linux-3.10-board-foo.tar.gz/

That's more or less what Thomas DS proposed, and I wasn't too happy
with it due to the horrible directory name. Another option would be to
hash the tarball location and do:

output/build/linux-${hash}/

but then it's difficult for the user to map which linux-<foo> directory
corresponds to which version he has built.

> > But I don't need all that and I now use --single-branch argument
> > to git clone followed by git archive to create a tar ball.
> 
>   That's a great idea! However, this option is only available since git 
> 1.7.10 and I think that many production environments still have older git 
> versions.

Can't we do like what we do for shallow clones? Try if --single-branch
is supported and works, and if so, all right, if not, fall back to the
normal way?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] feature or pebkac

[Arnout Vandecappelle]

On 08/30/13 09:58, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
>
> On Thu, 29 Aug 2013 21:05:53 +0200, Arnout Vandecappelle wrote:
>> On 08/29/13 20:17, rh wrote:
[snip]
>>> But I don't need all that and I now use --single-branch argument
>>> to git clone followed by git archive to create a tar ball.
>>
>>    That's a great idea! However, this option is only available since git
>> 1.7.10 and I think that many production environments still have older git
>> versions.
>
> Can't we do like what we do for shallow clones? Try if --single-branch
> is supported and works, and if so, all right, if not, fall back to the
> normal way?

  Not so easily. For shallow clone, we check up front with ls-remote if 
it will be possible.  For --single-branch we would have to check the git 
version. And the download commands already are too complex...

  I'm really thinking we should move all that to a proper shell script. 
Then adding that kind of magic would feel as much less of an overhead.


  Regards,
  Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] feature or pebkac

[Thomas Petazzoni]

Dear Arnout Vandecappelle,

On Fri, 30 Aug 2013 10:22:26 +0200, Arnout Vandecappelle wrote:

> > Can't we do like what we do for shallow clones? Try if --single-branch
> > is supported and works, and if so, all right, if not, fall back to the
> > normal way?
> 
>   Not so easily. For shallow clone, we check up front with ls-remote if 
> it will be possible.  For --single-branch we would have to check the git 
> version. And the download commands already are too complex...
> 
>   I'm really thinking we should move all that to a proper shell script. 
> Then adding that kind of magic would feel as much less of an overhead.

Yes, agreed.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] feature or pebkac

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 Arnout>  I'm really thinking we should move all that to a proper shell
 Arnout> script. Then adding that kind of magic would feel as much less of an
 Arnout> overhead.

Agreed.

-- 
Bye, Peter Korsgaard