[Buildroot] Two stages compilation

[Buildroot] Two stages compilation

[Angelo Compagnucci]

Hi List,

I'm interested in porting mono package on buildroot. I aim to
inclusion and offering to maintain it in the future, but I need a
little bit of help.

Mono compilation is in two stages:

1) Mono compiles the runtime for the target architecture. It means
that in this step you produce an ARM mono naked virtual machine.
2) Mono compiles mono assemblies (c# libraries). This phase fails on
my package, cause to compile assemblies you should have Mono for the
host architecture, not the target one. So to compile assemblies you
should use the x86 mono's version installed on your computer.
Alternatively, if you don't want to install mono on your computer, the
makefile can download a precompiled stripped version just to compile
the assemblies, so the first phase is optional.

I have the first phase running, so right now I can produce the ARM
mono virtual machine correctly, than I have to manual recompile
assemblies and transfer them to the target. I really want to make
everything as automatic as possible.

So in the second phase I should force buildroot to compile mono on the
host environment and not the target one. How can I do this?

Which is the advisable way to get the patch accepted?

Thank you for your time!

Angelo

-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] Two stages compilation

[Zoltan Gyarmati]

Dear Angelo,

The point is here that for mono you have to define both a host package,
and a target package. For this you can look for the
"host-autotools-package" macro, and probably can get some ideas from
checking package/python/python.mk.

A couple of months back i was able to package mono for OpenWrt (which is
in this sense close to Buildroot), but i haven't finalized that project
(as it turned out that the target MIPS CPU is not totally supported by
Mono...). If you think it's useful, i can dig this out from the archive
and share the package definition with you. Please let me know if so.

regards
zgyarmati



On 29.04.2014 11:29, Angelo Compagnucci wrote:
> Hi List,
>
> I'm interested in porting mono package on buildroot. I aim to
> inclusion and offering to maintain it in the future, but I need a
> little bit of help.
>
> Mono compilation is in two stages:
>
> 1) Mono compiles the runtime for the target architecture. It means
> that in this step you produce an ARM mono naked virtual machine.
> 2) Mono compiles mono assemblies (c# libraries). This phase fails on
> my package, cause to compile assemblies you should have Mono for the
> host architecture, not the target one. So to compile assemblies you
> should use the x86 mono's version installed on your computer.
> Alternatively, if you don't want to install mono on your computer, the
> makefile can download a precompiled stripped version just to compile
> the assemblies, so the first phase is optional.
>
> I have the first phase running, so right now I can produce the ARM
> mono virtual machine correctly, than I have to manual recompile
> assemblies and transfer them to the target. I really want to make
> everything as automatic as possible.
>
> So in the second phase I should force buildroot to compile mono on the
> host environment and not the target one. How can I do this?
>
> Which is the advisable way to get the patch accepted?
>
> Thank you for your time!
>
> Angelo
>


-- 
br, 
Zoltan Gyarmati
mail: mr.zoltan.gyarmati at gmail.com
freenode nick: zgyarmati

[Buildroot] Two stages compilation

[Angelo Compagnucci]

Hi Zoltan,

Thank you very much for the info, I'm reading the documentation avidly
but I totally missed that option.

Could you please send me the package definitions you mentioned above
as an example?

Thank you!

2014-04-29 12:53 GMT+02:00 Zoltan Gyarmati <mr.zoltan.gyarmati@gmail.com>:
> Dear Angelo,
>
> The point is here that for mono you have to define both a host package,
> and a target package. For this you can look for the
> "host-autotools-package" macro, and probably can get some ideas from
> checking package/python/python.mk.
>
> A couple of months back i was able to package mono for OpenWrt (which is
> in this sense close to Buildroot), but i haven't finalized that project
> (as it turned out that the target MIPS CPU is not totally supported by
> Mono...). If you think it's useful, i can dig this out from the archive
> and share the package definition with you. Please let me know if so.
>
> regards
> zgyarmati
>
>
>
> On 29.04.2014 11:29, Angelo Compagnucci wrote:
>> Hi List,
>>
>> I'm interested in porting mono package on buildroot. I aim to
>> inclusion and offering to maintain it in the future, but I need a
>> little bit of help.
>>
>> Mono compilation is in two stages:
>>
>> 1) Mono compiles the runtime for the target architecture. It means
>> that in this step you produce an ARM mono naked virtual machine.
>> 2) Mono compiles mono assemblies (c# libraries). This phase fails on
>> my package, cause to compile assemblies you should have Mono for the
>> host architecture, not the target one. So to compile assemblies you
>> should use the x86 mono's version installed on your computer.
>> Alternatively, if you don't want to install mono on your computer, the
>> makefile can download a precompiled stripped version just to compile
>> the assemblies, so the first phase is optional.
>>
>> I have the first phase running, so right now I can produce the ARM
>> mono virtual machine correctly, than I have to manual recompile
>> assemblies and transfer them to the target. I really want to make
>> everything as automatic as possible.
>>
>> So in the second phase I should force buildroot to compile mono on the
>> host environment and not the target one. How can I do this?
>>
>> Which is the advisable way to get the patch accepted?
>>
>> Thank you for your time!
>>
>> Angelo
>>
>
>
> --
> br,
> Zoltan Gyarmati
> mail: mr.zoltan.gyarmati at gmail.com
> freenode nick: zgyarmati
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] Two stages compilation

[Thomas De Schampheleire]

Hi Angelo, Zoltan,

(please don't top-post)

On Wed, Apr 30, 2014 at 9:30 AM, Angelo Compagnucci
<angelo.compagnucci@gmail.com> wrote:
> Hi Zoltan,
>
> Thank you very much for the info, I'm reading the documentation avidly
> but I totally missed that option.
>
> Could you please send me the package definitions you mentioned above
> as an example?
>
> Thank you!
>
> 2014-04-29 12:53 GMT+02:00 Zoltan Gyarmati <mr.zoltan.gyarmati@gmail.com>:
>> Dear Angelo,
>>
>> The point is here that for mono you have to define both a host package,
>> and a target package. For this you can look for the
>> "host-autotools-package" macro, and probably can get some ideas from
>> checking package/python/python.mk.
>>
>> A couple of months back i was able to package mono for OpenWrt (which is
>> in this sense close to Buildroot), but i haven't finalized that project
>> (as it turned out that the target MIPS CPU is not totally supported by
>> Mono...). If you think it's useful, i can dig this out from the archive
>> and share the package definition with you. Please let me know if so.

Note that Alexander Varnin has submitted a patch to the list for the
mono runtime a while back:
http://patchwork.ozlabs.org/patch/299488/

Best regards,
Thomas

[Buildroot] Two stages compilation

[Thomas Petazzoni]

Thomas, Angelo,

On Wed, 30 Apr 2014 20:44:57 +0200, Thomas De Schampheleire wrote:

> >> A couple of months back i was able to package mono for OpenWrt (which is
> >> in this sense close to Buildroot), but i haven't finalized that project
> >> (as it turned out that the target MIPS CPU is not totally supported by
> >> Mono...). If you think it's useful, i can dig this out from the archive
> >> and share the package definition with you. Please let me know if so.
> 
> Note that Alexander Varnin has submitted a patch to the list for the
> mono runtime a while back:
> http://patchwork.ozlabs.org/patch/299488/

Though this particular package only provides a target version of Mono,
which probably means that it relies on mono being installed on the
build machine to be able to build all the Mono C# libraries (i.e step 2
in Angelo's description).

So indeed, as Zoltan pointed out, most likely the solution is to have
both a target and a host variant for Mono. The host variant builds the
compiler, and is used on the build machine to compile all the Mono C#
libraries into the corresponding byte code. The target variant builds
the "virtual machine" that runs on the target. And obviously most
likely the target variant will have a dependency on the host variant.

Also, the host variant building the compiler will be useful if people
want to package other Mono libraries/applications.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Two stages compilation

[Angelo Compagnucci]

HiTtomas

>> Note that Alexander Varnin has submitted a patch to the list for the
>> mono runtime a while back:
>> http://patchwork.ozlabs.org/patch/299488/
>
> Though this particular package only provides a target version of Mono,
> which probably means that it relies on mono being installed on the
> build machine to be able to build all the Mono C# libraries (i.e step 2
> in Angelo's description).

Yes, that patch only provides the first half, I want to provide both the halves.

> So indeed, as Zoltan pointed out, most likely the solution is to have
> both a target and a host variant for Mono. The host variant builds the
> compiler, and is used on the build machine to compile all the Mono C#
> libraries into the corresponding byte code. The target variant builds
> the "virtual machine" that runs on the target. And obviously most
> likely the target variant will have a dependency on the host variant.
>
> Also, the host variant building the compiler will be useful if people
> want to package other Mono libraries/applications.

Yes, this is exactly what I aiming at. Actually I have a mono
metapackage that imports two subpackages, mono-native and
mono-managed.
The mono native part works like a charm, the mono-managed is where I'm
having troubles.

Actually I have something like this:


#############################################################
#
# mono-managed
#
#############################################################

MONO_MANAGED_VERSION         = $(MONO_VERSION)
MONO_MANAGED_SITE            = $(MONO_SITE)
MONO_MANAGED_SOURCE          = $(MONO_SOURCE)
MONO_MANAGED_INSTALL_STAGING  = $(MONO_INSTALL_STAGING)

HOST_MONO_MANAGED_CONF_ENV = \
CFLAGS="--sysroot=$(STAGING_DIR)"

HOST_MONO_MANAGED_CONF_OPT = \
$(MONO_CONF_OPT)

$(eval $(autotools-package))
$(eval $(host-autotools-package))


But looking at the configure phase I can see:

   Libraries:
        .NET 2.0/3.5:  no
        .NET 4.0:      no
        .NET 4.5:      no
        MonoDroid:     no
        MonoTouch:     no
        Xamarin.Mac:   no
        JNI support:   IKVM Native
        libgdiplus:    assumed to be installed
        zlib:

So .NET libraries will be not compiled.

If I enter manually the the build/mono-managed folder and issue a
./configure I can see this:

 Libraries:
        .NET 2.0/3.5:  yes
        .NET 4.0:      yes
        .NET 4.5:      yes
        MonoDroid:     no
        MonoTouch:     no
        Xamarin.Mac:   no
        JNI support:   IKVM Native
        libgdiplus:    assumed to be installed
        zlib:          system zlib

The main difference I can see in both config.log is --target and
--host options, they are (wrongly?)

--target=arm-buildroot-linux-uclibcgnueabi
--host=arm-buildroot-linux-uclibcgnueabi

when they should be

--host=x86_64-linux-gnu
--target=x86_64-linux-gnu

Have you any advice?

Thank you!

>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux, Kernel and Android engineering
> http://free-electrons.com



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] Two stages compilation

[Thomas De Schampheleire]

Hi Angelo,

Angelo Compagnucci <angelo.compagnucci@gmail.com> schreef:
>HiTtomas
>
>>> Note that Alexander Varnin has submitted a patch to the list for the
>>> mono runtime a while back:
>>> http://patchwork.ozlabs.org/patch/299488/
>>
>> Though this particular package only provides a target version of Mono,
>> which probably means that it relies on mono being installed on the
>> build machine to be able to build all the Mono C# libraries (i.e step 2
>> in Angelo's description).
>
>Yes, that patch only provides the first half, I want to provide both the halves.
>
>> So indeed, as Zoltan pointed out, most likely the solution is to have
>> both a target and a host variant for Mono. The host variant builds the
>> compiler, and is used on the build machine to compile all the Mono C#
>> libraries into the corresponding byte code. The target variant builds
>> the "virtual machine" that runs on the target. And obviously most
>> likely the target variant will have a dependency on the host variant.
>>
>> Also, the host variant building the compiler will be useful if people
>> want to package other Mono libraries/applications.
>
>Yes, this is exactly what I aiming at. Actually I have a mono
>metapackage that imports two subpackages, mono-native and
>mono-managed.
>The mono native part works like a charm, the mono-managed is where I'm
>having troubles.
>
>Actually I have something like this:
>
>
>#############################################################
>#
># mono-managed
>#
>#############################################################
>
>MONO_MANAGED_VERSION         = $(MONO_VERSION)
>MONO_MANAGED_SITE            = $(MONO_SITE)
>MONO_MANAGED_SOURCE          = $(MONO_SOURCE)
>MONO_MANAGED_INSTALL_STAGING  = $(MONO_INSTALL_STAGING)
>
>HOST_MONO_MANAGED_CONF_ENV = \
>CFLAGS="--sysroot=$(STAGING_DIR)"
>
>HOST_MONO_MANAGED_CONF_OPT = \
>$(MONO_CONF_OPT)
>
>$(eval $(autotools-package))
>$(eval $(host-autotools-package))
>
>
>But looking at the configure phase I can see:
>
>   Libraries:
>        .NET 2.0/3.5:  no
>        .NET 4.0:      no
>        .NET 4.5:      no
>        MonoDroid:     no
>        MonoTouch:     no
>        Xamarin.Mac:   no
>        JNI support:   IKVM Native
>        libgdiplus:    assumed to be installed
>        zlib:
>
>So .NET libraries will be not compiled.
>
>If I enter manually the the build/mono-managed folder and issue a
>./configure I can see this:
>
> Libraries:
>        .NET 2.0/3.5:  yes
>        .NET 4.0:      yes
>        .NET 4.5:      yes
>        MonoDroid:     no
>        MonoTouch:     no
>        Xamarin.Mac:   no
>        JNI support:   IKVM Native
>        libgdiplus:    assumed to be installed
>        zlib:          system zlib
>
>The main difference I can see in both config.log is --target and
>--host options, they are (wrongly?)
>
>--target=arm-buildroot-linux-uclibcgnueabi
>--host=arm-buildroot-linux-uclibcgnueabi
>
>when they should be
>
>--host=x86_64-linux-gnu
>--target=x86_64-linux-gnu
>
>Have you any advice?

If I understand the situation correctly, you only need the managed part for host, right? In this case, you only need the host-autotools line, not the autotools one.

Also, I think you should provide all what you have, plus a bigger part of the log.
I believe the log you showed was from the 'target' managed part, caused by the fact you still had the normal autotools line.
You should be looking at the output for host-mono-managed.

Since host packages are normally not user selectable, you need to add a dependency to it from another package. For example using:

MONO_NATIVE_DEPENDENCIES += host-mono-managed

Another approach is where the use can choose whether he wants the host part too. This is only good if course if it is not strictly required. In this case you would add it in a Config.in.host file which would be included from the host utilities menu.

Best regards,
Thomas

[Buildroot] Two stages compilation

[Angelo Compagnucci]

> Since host packages are normally not user selectable, you need to add a dependency to it from another package. For example using:
>
> MONO_NATIVE_DEPENDENCIES += host-mono-managed
>
> Another approach is where the use can choose whether he wants the host part too. This is only good if course if it is not strictly required. In this case you would add it in a Config.in.host file which would be included from the host utilities menu.

I Thomas, after your advice, everything started working!!! Right now
I'm making some more tests, but the packages compiles correctly. When
finished, I'll make a patch and publish it on the mailing list.

Probably I have to reread the documentation because I missed this part
about host compiling and dependencies.

Thank you again for your support!


-- 
Profile: http://it.linkedin.com/in/compagnucciangelo