[Buildroot] Is there an equivalent of yocto's devshell?

[Buildroot] Is there an equivalent of yocto's devshell?

[Ottavio Campana]

Hello,

I'm quite new to buildroot, and I am trying to check how I can use it in one project. I am very familiar with yocto, so I am trying to re-learn how to do things with buildroot.

Particularly, I am not able to find if buildroot offers a feature similar to the devshell of yocto. The devshell is a shell with all the environment variables already set, so that it is possibile to invoke commands such as make in the same way that the build process does. By working in the devshell, a developer can work already in the correct environment for cross-compiling, which can save a lot of time.

Does something similar to the devshell exist in buildroot?

Thank you

Ottavio

[Buildroot] Is there an equivalent of yocto's devshell?

[Peter Korsgaard]

>>>>> "Ottavio" == Ottavio Campana <o.campana@videotec.com> writes:

 > Hello,
 > I'm quite new to buildroot, and I am trying to check how I can use it
 > in one project. I am very familiar with yocto, so I am trying to
 > re-learn how to do things with buildroot.

 > Particularly, I am not able to find if buildroot offers a feature
 > similar to the devshell of yocto. The devshell is a shell with all the
 > environment variables already set, so that it is possibile to invoke
 > commands such as make in the same way that the build process does. By
 > working in the devshell, a developer can work already in the correct
 > environment for cross-compiling, which can save a lot of time.

 > Does something similar to the devshell exist in buildroot?

I have never used this 'devshell', but I take it that it is something
that adds the cross compiler to the path and possibly sets up some
environment variables like CC/CXX/CROSS_COMPILE/ARCH to point to the
cross compiler?

If so, we don't have anything quite like that, but writing a little
shell script for your project adding output/host/usr/bin to your path
and exporting those environment variables shouldn't take many minutes.

Or does it do something more special than that?

-- 
Bye, Peter Korsgaard

[Buildroot] Is there an equivalent of yocto's devshell?

[Ottavio Campana]

> > Particularly, I am not able to find if buildroot offers a feature
> > similar to the devshell of yocto. The devshell is a shell with all the
> > environment variables already set, so that it is possibile to invoke
> > commands such as make in the same way that the build process does. By
> > working in the devshell, a developer can work already in the correct
> > environment for cross-compiling, which can save a lot of time.
>
> > Does something similar to the devshell exist in buildroot?
>
> I have never used this 'devshell', but I take it that it is something
> that adds the cross compiler to the path and possibly sets up some
> environment variables like CC/CXX/CROSS_COMPILE/ARCH to point to the
> cross compiler?
>
> If so, we don't have anything quite like that, but writing a little
> shell script for your project adding output/host/usr/bin to your path
> and exporting those environment variables shouldn't take many minutes.
>
> Or does it do something more special than that?

Well, le'ts say that it configures the not only the cross compiler but it also sets all the variables for pointing to the staging environment for finding includes and linking to libraries. Compiling a self contained program is trivial, but when you need to include different libraries things start to be more difficult.

I can write the script, but can you tell my where I can find all the variables that are set by buildroot before compiling a package?

Thank you

Ottavio

[Buildroot] Is there an equivalent of yocto's devshell?

[Matt Flax]

On 01/09/16 17:20, Ottavio Campana wrote:
>>> Particularly, I am not able to find if buildroot offers a feature
>>> similar to the devshell of yocto. The devshell is a shell with all the
>>> environment variables already set, so that it is possibile to invoke
>>> commands such as make in the same way that the build process does. By
>>> working in the devshell, a developer can work already in the correct
>>> environment for cross-compiling, which can save a lot of time.
>>> Does something similar to the devshell exist in buildroot?
>> I have never used this 'devshell', but I take it that it is something
>> that adds the cross compiler to the path and possibly sets up some
>> environment variables like CC/CXX/CROSS_COMPILE/ARCH to point to the
>> cross compiler?
>>
>> If so, we don't have anything quite like that, but writing a little
>> shell script for your project adding output/host/usr/bin to your path
>> and exporting those environment variables shouldn't take many minutes.
>>
>> Or does it do something more special than that?
> Well, le'ts say that it configures the not only the cross compiler but it also sets all the variables for pointing to the staging environment for finding includes and linking to libraries. Compiling a self contained program is trivial, but when you need to include different libraries things start to be more difficult.
>
> I can write the script, but can you tell my where I can find all the variables that are set by buildroot before compiling a package?
>
You can find that information out by echoing env at some point in the 
package build process.

Is it necessary to have a devshell ? Could you simply run 'make package' 
instead and work live in the output/build/package directory ?
You can rerun various parts of the make process by deleting the .stamp_* 
files in the package directory.

Conceptually if you build the package you are working on natively on 
your desktop, you should be able to do all the library inclusion and 
linking locally ... once happy, you can simply invoke buildroot to cross 
compile for you ... I would imagine that that would be a much faster way 
top develop ...

Matt

[Buildroot] Is there an equivalent of yocto's devshell?

[Thomas Petazzoni]

Hello,

On Thu, 1 Sep 2016 17:51:31 +1000, Matt Flax wrote:

> Is it necessary to have a devshell ? Could you simply run 'make package' 
> instead and work live in the output/build/package directory ?
> You can rerun various parts of the make process by deleting the .stamp_* 
> files in the package directory.
> 
> Conceptually if you build the package you are working on natively on 
> your desktop, you should be able to do all the library inclusion and 
> linking locally ... once happy, you can simply invoke buildroot to cross 
> compile for you ... I would imagine that that would be a much faster way 
> top develop ...

Fully agreed. I am not sure I understand the usefulness of devshell. It
is just easier and more efficient to use the build system directly.

However, if you're actively working on the source of a package, do
*NOT* make your changes in outptu/build/<package>-<version>/. This
directory is a temporary location. Instead, use the OVERRIDE_SRCDIR
mechanism, which allows you to tell Buildroot to take the source code
for a given package from a given directory. See
http://free-electrons.com/doc/training/buildroot/buildroot-slides.pdf
slides 261 and following.

My feeling is that things like devshell are created because using Yocto
during development is too painful. Buildroot simply doesn't need
something like devshell because using the build system during active
development is perfectly fine. Of course, if you disagree and have some
concrete examples, we'll be happy to look into this.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] Is there an equivalent of yocto's devshell?

[Zoltan Gyarmati]

Dear Ottavio & All,

just as a sidenote: if you happen to work with qmake based project, you
can easily just call the qmake from the host dir
(buildroot/output/host/usr/bin/qmake) dirclty from your source dir, and
the generated Makefile will use the Buildroot environment, sets the
correct include and link flags, for libraries contained by Buildroot as
well. Of course things are getting a bit complicated if you are
developing your lib and your application simultaneously, then better to
use the mentioned source dir overriding mechanism.


Regards
zgyarmati


On 2016-09-01 11:39, Thomas Petazzoni wrote:
> Hello,
>
> On Thu, 1 Sep 2016 17:51:31 +1000, Matt Flax wrote:
>
>> Is it necessary to have a devshell ? Could you simply run 'make package' 
>> instead and work live in the output/build/package directory ?
>> You can rerun various parts of the make process by deleting the .stamp_* 
>> files in the package directory.
>>
>> Conceptually if you build the package you are working on natively on 
>> your desktop, you should be able to do all the library inclusion and 
>> linking locally ... once happy, you can simply invoke buildroot to cross 
>> compile for you ... I would imagine that that would be a much faster way 
>> top develop ...
> Fully agreed. I am not sure I understand the usefulness of devshell. It
> is just easier and more efficient to use the build system directly.
>
> However, if you're actively working on the source of a package, do
> *NOT* make your changes in outptu/build/<package>-<version>/. This
> directory is a temporary location. Instead, use the OVERRIDE_SRCDIR
> mechanism, which allows you to tell Buildroot to take the source code
> for a given package from a given directory. See
> http://free-electrons.com/doc/training/buildroot/buildroot-slides.pdf
> slides 261 and following.
>
> My feeling is that things like devshell are created because using Yocto
> during development is too painful. Buildroot simply doesn't need
> something like devshell because using the build system during active
> development is perfectly fine. Of course, if you disagree and have some
> concrete examples, we'll be happy to look into this.
>
> Best regards,
>
> Thomas


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160901/cba068a2/attachment.asc>

[Buildroot] Is there an equivalent of yocto's devshell?

[Ottavio Campana]

> My feeling is that things like devshell are created because using Yocto
> during development is too painful. Buildroot simply doesn't need
> something like devshell because using the build system during active
> development is perfectly fine. Of course, if you disagree and have some
> concrete examples, we'll be happy to look into this.

Thomas, you are 100%.

The usual situation is you develop a portable program, but then you need to specialize some part that is platform-specify and you need to work on the sources with the cross compiler.

I think I'll echo the environment and then reproduce it when I'll work on the sources.

Thank you

Ottavio

[Buildroot] Is there an equivalent of yocto's devshell?

[Arnout Vandecappelle]

On 01-09-16 09:20, Ottavio Campana wrote:
>>> Particularly, I am not able to find if buildroot offers a feature
>>> similar to the devshell of yocto. The devshell is a shell with all the
>>> environment variables already set, so that it is possibile to invoke
>>> commands such as make in the same way that the build process does. By
>>> working in the devshell, a developer can work already in the correct
>>> environment for cross-compiling, which can save a lot of time.
>>
>>> Does something similar to the devshell exist in buildroot?
>>
>> I have never used this 'devshell', but I take it that it is something
>> that adds the cross compiler to the path and possibly sets up some
>> environment variables like CC/CXX/CROSS_COMPILE/ARCH to point to the
>> cross compiler?
>>
>> If so, we don't have anything quite like that, but writing a little
>> shell script for your project adding output/host/usr/bin to your path
>> and exporting those environment variables shouldn't take many minutes.
>>
>> Or does it do something more special than that?
> 
> Well, le'ts say that it configures the not only the cross compiler but it also sets all the variables for pointing to the staging environment for finding includes and linking to libraries. Compiling a self contained program is trivial, but when you need to include different libraries things start to be more difficult.

 The toolchain is wrapped to make sure it points to the correct sysroot. So just
calling output/usr/bin/<cross>-gcc is enough. Same for pkg-config:
output/usr/bin/pkg-config is a wrapper script that sets the correct environment
variables.

 There is still a pretty long list of variables that buildroot passes down to
the build system. You can find it in package/Makefile.in:

TARGET_CONFIGURE_OPTS = \
        PATH=$(BR_PATH) \
        AR="$(TARGET_AR)" \
        AS="$(TARGET_AS)" \
        LD="$(TARGET_LD)" \
        NM="$(TARGET_NM)" \
        CC="$(TARGET_CC)" \
        GCC="$(TARGET_CC)" \
        CPP="$(TARGET_CPP)" \
        CXX="$(TARGET_CXX)" \
        FC="$(TARGET_FC)" \
        F77="$(TARGET_FC)" \
        RANLIB="$(TARGET_RANLIB)" \
        READELF="$(TARGET_READELF)" \
        STRIP="$(TARGET_STRIP)" \
        OBJCOPY="$(TARGET_OBJCOPY)" \
        OBJDUMP="$(TARGET_OBJDUMP)" \
        AR_FOR_BUILD="$(HOSTAR)" \
        AS_FOR_BUILD="$(HOSTAS)" \
        CC_FOR_BUILD="$(HOSTCC)" \
        GCC_FOR_BUILD="$(HOSTCC)" \
        CXX_FOR_BUILD="$(HOSTCXX)" \
        LD_FOR_BUILD="$(HOSTLD)" \
        CPPFLAGS_FOR_BUILD="$(HOST_CPPFLAGS)" \
        CFLAGS_FOR_BUILD="$(HOST_CFLAGS)" \
        CXXFLAGS_FOR_BUILD="$(HOST_CXXFLAGS)" \
        LDFLAGS_FOR_BUILD="$(HOST_LDFLAGS)" \
        FCFLAGS_FOR_BUILD="$(HOST_FCFLAGS)" \
        DEFAULT_ASSEMBLER="$(TARGET_AS)" \
        DEFAULT_LINKER="$(TARGET_LD)" \
        CPPFLAGS="$(TARGET_CPPFLAGS)" \
        CFLAGS="$(TARGET_CFLAGS)" \
        CXXFLAGS="$(TARGET_CXXFLAGS)" \
        LDFLAGS="$(TARGET_LDFLAGS)" \
        FCFLAGS="$(TARGET_FCFLAGS)" \
        FFLAGS="$(TARGET_FCFLAGS)" \
        PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
        STAGING_DIR="$(STAGING_DIR)" \
        INTLTOOL_PERL=$(PERL)

 Obviously, you won't need the majority of them for a particular build.

 Whenever you build a package based on autotools or make, you'll see that
environment appearing in the build output, so you can cut and paste from there.
Alternatively, you can run 'make printvars' and extract the
TARGET_CONFIGURE_OPTS environment variable.

 As Zoltan mentioned further down, qmake-based packages can directly use
output/host/usr/bin/qmake. For cmake-based packages, use
cmake -DCMAKE_TOOLCHAIN_FILE="output/host/usr/share/buildroot/toolchainfile.cmake"

 However, as Thomas explains further down, it's really a lot simpler to make a
package/foo/foo.mk file. If all you want to do is to build and not bother with
installation or dependencies or anything like that, it can be really simple:

define FOO_BUILD_CMDS
	$(TARGET_CONFIGURE_OPTS) $(MAKE) -C $(@D)
endef
$(eval $(generic-package))

That should be enough, and you can build with:

make foo FOO_OVERRIDE_SRCDIR=path/to/src

 You don't even have to create a different foo.mk for different packages - but
make sure you clean up the build directory between builds.


 Regards,
 Arnout


> 
> I can write the script, but can you tell my where I can find all the variables that are set by buildroot before compiling a package?
> 
> Thank you
> 
> Ottavio
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Is there an equivalent of yocto's devshell?

[Matt Flax]

On 01/09/16 22:18, Ottavio Campana wrote:
>> My feeling is that things like devshell are created because using Yocto
>> during development is too painful. Buildroot simply doesn't need
>> something like devshell because using the build system during active
>> development is perfectly fine. Of course, if you disagree and have some
>> concrete examples, we'll be happy to look into this.
> Thomas, you are 100%.
>
> The usual situation is you develop a portable program, but then you need to specialize some part that is platform-specify and you need to work on the sources with the cross compiler.
>
>
Further on this topic...
If you need to work on sources w.r.t. specific cross compile setups, 
there are a few ways to do that effectively/efficiently when using 
buildroot.

One way for example is to use quilt ... e.g. when editing an old linux 
mach/plat device file, have an external buildroot directory with a 
patches/linux directory.
You can then "make linux" then "cd output/build/linux" and "ln -s" your 
external patches/linux folder to the quilt expected patches directory. 
You can then use quilt as expected, and whenever "quilt refresh" is run, 
the external patches get updated.

In this manner, you use standard buildroot and quilt to manage patches 
for cross-compiled packages ... once you are happy, you can then either 
use the patches as per the standard process with buildroot or apply them 
against your original sources.

Matt