[U-Boot] buildman with distcc

[U-Boot] buildman with distcc

[York Sun]

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling.
Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
don't know how to do this with buildman.

York

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 03:01 PM, York Sun wrote:
> Simon,
> 
> Is it possible to use buildman with distcc? I am trying to speed up compiling.
> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
> don't know how to do this with buildman.
> 

I meant to use buildman with multiple architectures, like 'arm & freescale'
powerpc aarch64 together.

York

[U-Boot] buildman with distcc

[Simon Glass]

Hi York,

On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>
> On 11/13/2014 03:01 PM, York Sun wrote:
> > Simon,
> >
> > Is it possible to use buildman with distcc? I am trying to speed up compiling.
> > Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
> > don't know how to do this with buildman.
> >
>
> I meant to use buildman with multiple architectures, like 'arm & freescale'
> powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section
of ~/.buildman ?

Regards,
Simon

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 04:01 PM, Simon Glass wrote:
> Hi York,
> 
> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>
>> On 11/13/2014 03:01 PM, York Sun wrote:
>>> Simon,
>>>
>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>> don't know how to do this with buildman.
>>>
>>
>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>> powerpc aarch64 together.
> 
> I wonder what would happen if you put it in the '[toolchain]' section
> of ~/.buildman ?
> 

buildman --list-tool-chains cannot detect the toolchain if doing so.

If not using buildman, I can modify Makefile to prefix distcc to CC and keep
using MAKEALL.

York

[U-Boot] buildman with distcc

[Simon Glass]

Hi York,

On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
> On 11/13/2014 04:01 PM, Simon Glass wrote:
>> Hi York,
>>
>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>
>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>> Simon,
>>>>
>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>> don't know how to do this with buildman.
>>>>
>>>
>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>> powerpc aarch64 together.
>>
>> I wonder what would happen if you put it in the '[toolchain]' section
>> of ~/.buildman ?
>>
>
> buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command
line. But that should not affect operation.

>
> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
> using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Regards,
Simon

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 04:14 PM, Simon Glass wrote:
> Hi York,
> 
> On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
>> On 11/13/2014 04:01 PM, Simon Glass wrote:
>>> Hi York,
>>>
>>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>>
>>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>>> Simon,
>>>>>
>>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>>> don't know how to do this with buildman.
>>>>>
>>>>
>>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>>> powerpc aarch64 together.
>>>
>>> I wonder what would happen if you put it in the '[toolchain]' section
>>> of ~/.buildman ?
>>>
>>
>> buildman --list-tool-chains cannot detect the toolchain if doing so.
> 
> No, it works by looking for filenames rather than running the command
> line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a
complete path. I need to put double quote for "distcc <my path prefix>". I don't
see how that could work.

> 
>>
>> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
>> using MAKEALL.
> 
> Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a
checkout.

York

[U-Boot] buildman with distcc

[Simon Glass]

Hi York,

On 13 November 2014 17:17, York Sun <yorksun@freescale.com> wrote:
> On 11/13/2014 04:14 PM, Simon Glass wrote:
>> Hi York,
>>
>> On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
>>> On 11/13/2014 04:01 PM, Simon Glass wrote:
>>>> Hi York,
>>>>
>>>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>>>
>>>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>>>> Simon,
>>>>>>
>>>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>>>> don't know how to do this with buildman.
>>>>>>
>>>>>
>>>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>>>> powerpc aarch64 together.
>>>>
>>>> I wonder what would happen if you put it in the '[toolchain]' section
>>>> of ~/.buildman ?
>>>>
>>>
>>> buildman --list-tool-chains cannot detect the toolchain if doing so.
>>
>> No, it works by looking for filenames rather than running the command
>> line. But that should not affect operation.
>
> The path in ./.buildman is partial. buildman appends other stuff to make a
> complete path. I need to put double quote for "distcc <my path prefix>". I don't
> see how that could work.

It doesn't have to - if you give a complete path then it will use it.
You might need to change things in toolchain.py to make it work. See
MakeEnvironment() which sets up the environment for Make -
CROSS_COMPILE, PATH.

Yes, one problem is that buildman uses objsize to find out the
code/data size. How can that work with distcc?

>
>>
>>>
>>> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
>>> using MAKEALL.
>>
>> Indeed. I wonder why that doesn't work with buildman?
>
> Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a
> checkout.

Well if you are using that feature out certainly can't use MAKEALL
since it doesn't support it. If you leave off the branch (-b) buildman
will build the current source and not check anything out.

Regards,
Simon

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 04:30 PM, Simon Glass wrote:
> Hi York,
> 
> On 13 November 2014 17:17, York Sun <yorksun@freescale.com> wrote:
>> On 11/13/2014 04:14 PM, Simon Glass wrote:
>>> Hi York,
>>>
>>> On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
>>>> On 11/13/2014 04:01 PM, Simon Glass wrote:
>>>>> Hi York,
>>>>>
>>>>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>>>>
>>>>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>>>>> Simon,
>>>>>>>
>>>>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>>>>> don't know how to do this with buildman.
>>>>>>>
>>>>>>
>>>>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>>>>> powerpc aarch64 together.
>>>>>
>>>>> I wonder what would happen if you put it in the '[toolchain]' section
>>>>> of ~/.buildman ?
>>>>>
>>>>
>>>> buildman --list-tool-chains cannot detect the toolchain if doing so.
>>>
>>> No, it works by looking for filenames rather than running the command
>>> line. But that should not affect operation.
>>
>> The path in ./.buildman is partial. buildman appends other stuff to make a
>> complete path. I need to put double quote for "distcc <my path prefix>". I don't
>> see how that could work.
> 
> It doesn't have to - if you give a complete path then it will use it.
> You might need to change things in toolchain.py to make it work. See
> MakeEnvironment() which sets up the environment for Make -
> CROSS_COMPILE, PATH.
> 
> Yes, one problem is that buildman uses objsize to find out the
> code/data size. How can that work with distcc?

Probably it won't.

> 
>>
>>>
>>>>
>>>> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
>>>> using MAKEALL.
>>>
>>> Indeed. I wonder why that doesn't work with buildman?
>>
>> Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a
>> checkout.
> 
> Well if you are using that feature out certainly can't use MAKEALL
> since it doesn't support it. If you leave off the branch (-b) buildman
> will build the current source and not check anything out.
> 

I like to use buildman to build all the commits.

Can I force CC with buildman?

York

[U-Boot] buildman with distcc

[Simon Glass]

Hi York,


On 13 November 2014 17:32, York Sun <yorksun@freescale.com> wrote:
> On 11/13/2014 04:30 PM, Simon Glass wrote:
>> Hi York,
>>
>> On 13 November 2014 17:17, York Sun <yorksun@freescale.com> wrote:
>>> On 11/13/2014 04:14 PM, Simon Glass wrote:
>>>> Hi York,
>>>>
>>>> On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
>>>>> On 11/13/2014 04:01 PM, Simon Glass wrote:
>>>>>> Hi York,
>>>>>>
>>>>>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>>>>>
>>>>>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>>>>>> Simon,
>>>>>>>>
>>>>>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>>>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>>>>>> don't know how to do this with buildman.
>>>>>>>>
>>>>>>>
>>>>>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>>>>>> powerpc aarch64 together.
>>>>>>
>>>>>> I wonder what would happen if you put it in the '[toolchain]' section
>>>>>> of ~/.buildman ?
>>>>>>
>>>>>
>>>>> buildman --list-tool-chains cannot detect the toolchain if doing so.
>>>>
>>>> No, it works by looking for filenames rather than running the command
>>>> line. But that should not affect operation.
>>>
>>> The path in ./.buildman is partial. buildman appends other stuff to make a
>>> complete path. I need to put double quote for "distcc <my path prefix>". I don't
>>> see how that could work.
>>
>> It doesn't have to - if you give a complete path then it will use it.
>> You might need to change things in toolchain.py to make it work. See
>> MakeEnvironment() which sets up the environment for Make -
>> CROSS_COMPILE, PATH.
>>
>> Yes, one problem is that buildman uses objsize to find out the
>> code/data size. How can that work with distcc?
>
> Probably it won't.

I wonder how the build copes with things like 'objcopy' then, which
are in the Makefile.

>
>>
>>>
>>>>
>>>>>
>>>>> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
>>>>> using MAKEALL.
>>>>
>>>> Indeed. I wonder why that doesn't work with buildman?
>>>
>>> Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a
>>> checkout.
>>
>> Well if you are using that feature out certainly can't use MAKEALL
>> since it doesn't support it. If you leave off the branch (-b) buildman
>> will build the current source and not check anything out.
>>
>
> I like to use buildman to build all the commits.
>
> Can I force CC with buildman?

If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

Regards,
Simon

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 04:35 PM, Simon Glass wrote:
> Hi York,
> 
> 
> On 13 November 2014 17:32, York Sun <yorksun@freescale.com> wrote:
>> On 11/13/2014 04:30 PM, Simon Glass wrote:
>>> Hi York,
>>>
>>> On 13 November 2014 17:17, York Sun <yorksun@freescale.com> wrote:
>>>> On 11/13/2014 04:14 PM, Simon Glass wrote:
>>>>> Hi York,
>>>>>
>>>>> On 13 November 2014 17:02, York Sun <yorksun@freescale.com> wrote:
>>>>>> On 11/13/2014 04:01 PM, Simon Glass wrote:
>>>>>>> Hi York,
>>>>>>>
>>>>>>> On 13 November 2014 16:03, York Sun <yorksun@freescale.com> wrote:
>>>>>>>>
>>>>>>>> On 11/13/2014 03:01 PM, York Sun wrote:
>>>>>>>>> Simon,
>>>>>>>>>
>>>>>>>>> Is it possible to use buildman with distcc? I am trying to speed up compiling.
>>>>>>>>> Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I
>>>>>>>>> don't know how to do this with buildman.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I meant to use buildman with multiple architectures, like 'arm & freescale'
>>>>>>>> powerpc aarch64 together.
>>>>>>>
>>>>>>> I wonder what would happen if you put it in the '[toolchain]' section
>>>>>>> of ~/.buildman ?
>>>>>>>
>>>>>>
>>>>>> buildman --list-tool-chains cannot detect the toolchain if doing so.
>>>>>
>>>>> No, it works by looking for filenames rather than running the command
>>>>> line. But that should not affect operation.
>>>>
>>>> The path in ./.buildman is partial. buildman appends other stuff to make a
>>>> complete path. I need to put double quote for "distcc <my path prefix>". I don't
>>>> see how that could work.
>>>
>>> It doesn't have to - if you give a complete path then it will use it.
>>> You might need to change things in toolchain.py to make it work. See
>>> MakeEnvironment() which sets up the environment for Make -
>>> CROSS_COMPILE, PATH.
>>>
>>> Yes, one problem is that buildman uses objsize to find out the
>>> code/data size. How can that work with distcc?
>>
>> Probably it won't.
> 
> I wonder how the build copes with things like 'objcopy' then, which
> are in the Makefile.
> 
>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> If not using buildman, I can modify Makefile to prefix distcc to CC and keep
>>>>>> using MAKEALL.
>>>>>
>>>>> Indeed. I wonder why that doesn't work with buildman?
>>>>
>>>> Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a
>>>> checkout.
>>>
>>> Well if you are using that feature out certainly can't use MAKEALL
>>> since it doesn't support it. If you leave off the branch (-b) buildman
>>> will build the current source and not check anything out.
>>>
>>
>> I like to use buildman to build all the commits.
>>
>> Can I force CC with buildman?
> 
> If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But
I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50
jobs, the distribution is mostly to localhost. Not sure how buildman calls make.
Can you point me to the right direction?

York

[U-Boot] buildman with distcc

[York Sun]

On 11/13/2014 05:10 PM, York Sun wrote:
> On 11/13/2014 04:35 PM, Simon Glass wrote:
>>
>> If you add that feature you can. But maybe you just need to force CROSS_COMPILE?
> 
> I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But
> I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50
> jobs, the distribution is mostly to localhost. Not sure how buildman calls make.
> Can you point me to the right direction?
> 

Nah, don't bother. I tried a build. It doesn't speed up enough to pursue this
way, even with 6 servers. I may pick up in the future if I have to do this way.

York

[U-Boot] buildman with distcc

[Simon Glass]

Hi York,

On 14 November 2014 02:01, York Sun <yorksun@freescale.com> wrote:
> On 11/13/2014 05:10 PM, York Sun wrote:
>> On 11/13/2014 04:35 PM, Simon Glass wrote:
>>>
>>> If you add that feature you can. But maybe you just need to force CROSS_COMPILE?
>>
>> I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But
>> I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50
>> jobs, the distribution is mostly to localhost. Not sure how buildman calls make.
>> Can you point me to the right direction?
>>
>
> Nah, don't bother. I tried a build. It doesn't speed up enough to pursue this
> way, even with 6 servers. I may pick up in the future if I have to do this way.

OK.

Regards,
Simon