[Buildroot] [PATCH 1/2] jq: disable maintainer mode

[Buildroot] [PATCH 1/2] jq: disable maintainer mode

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Since version 1.5, maintainer mode is enabled by default. We don't need
this, and is in particular problematic because it requires bison 3.0.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
 package/jq/jq.mk | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/package/jq/jq.mk b/package/jq/jq.mk
index dd13276..8053e39 100644
--- a/package/jq/jq.mk
+++ b/package/jq/jq.mk
@@ -12,4 +12,7 @@ JQ_LICENSE_FILES = COPYING
 # uses c99 specific features
 JQ_CONF_ENV += CFLAGS="$(TARGET_CFLAGS) -std=c99"
 
+# jq explicitly enables maintainer mode, which we don't need/want
+JQ_CONF_OPTS += --disable-maintainer-mode
+
 $(eval $(autotools-package))
-- 
1.9.5

[Buildroot] [PATCH 2/2] jq: enable host package

[Thomas De Schampheleire]

From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Allow building jq as host utility for use in post-build scripts.
This can be useful to created, edit, merge or even perform syntax checking
on JSON files.

Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
---
 package/Config.in.host    |  1 +
 package/jq/Config.in.host | 11 +++++++++++
 package/jq/jq.mk          |  3 +++
 3 files changed, 15 insertions(+)
 create mode 100644 package/jq/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 1f69687..ce1b6bc 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -13,6 +13,7 @@ menu "Host utilities"
 	source "package/genimage/Config.in.host"
 	source "package/genpart/Config.in.host"
 	source "package/imx-usb-loader/Config.in.host"
+	source "package/jq/Config.in.host"
 	source "package/lpc3250loader/Config.in.host"
 	source "package/mke2img/Config.in.host"
 	source "package/mtd/Config.in.host"
diff --git a/package/jq/Config.in.host b/package/jq/Config.in.host
new file mode 100644
index 0000000..a0aabc0
--- /dev/null
+++ b/package/jq/Config.in.host
@@ -0,0 +1,11 @@
+config BR2_PACKAGE_HOST_JQ
+	bool "host jq"
+	help
+	  jq is like sed for JSON data - you can use it to slice and filter
+	  and map and transform structured data with the same ease that sed,
+	  awk, grep and friends let you play with text.
+
+	  As a host tool, jq can be useful to create/edit/merge/check JSON
+	  files from a post-build script.
+
+	  http://stedolan.github.io/jq/
diff --git a/package/jq/jq.mk b/package/jq/jq.mk
index 8053e39..9a094ba 100644
--- a/package/jq/jq.mk
+++ b/package/jq/jq.mk
@@ -11,8 +11,11 @@ JQ_LICENSE_FILES = COPYING
 
 # uses c99 specific features
 JQ_CONF_ENV += CFLAGS="$(TARGET_CFLAGS) -std=c99"
+HOST_JQ_CONF_ENV += CFLAGS="$(HOST_CFLAGS) -std=c99"
 
 # jq explicitly enables maintainer mode, which we don't need/want
 JQ_CONF_OPTS += --disable-maintainer-mode
+HOST_JQ_CONF_OPTS += --disable-maintainer-mode
 
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
1.9.5

[Buildroot] [PATCH 2/2] jq: enable host package

[Thomas Petazzoni]

Thomas,

On Mon, 19 Oct 2015 12:42:09 +0200, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> Allow building jq as host utility for use in post-build scripts.
> This can be useful to created, edit, merge or even perform syntax checking
> on JSON files.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

Since this host package clearly doesn't fall into the "debugging tool,
flashing tool or image preparation tool" category, I'd like to collect
the opinion of a few other developers before applying.

Peter, Arnout, Yann, what do you think?

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2] jq: enable host package

[Arnout Vandecappelle]

On 21-10-15 22:39, Thomas Petazzoni wrote:
> Thomas,
> 
> On Mon, 19 Oct 2015 12:42:09 +0200, Thomas De Schampheleire wrote:
>> From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>>
>> Allow building jq as host utility for use in post-build scripts.
>> This can be useful to created, edit, merge or even perform syntax checking
>> on JSON files.
>>
>> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> Since this host package clearly doesn't fall into the "debugging tool,
> flashing tool or image preparation tool" category, I'd like to collect
> the opinion of a few other developers before applying.
> 
> Peter, Arnout, Yann, what do you think?

 I was thinking the same thing when I say the patch, but I don't care enough to
really comment on it. It's a bit similar to packages that depend on some host
preprocessing tool, like flex. But then of course we just have a dependency and
not a Config.in.host.

 OTOH, if we accept this, it kind of opens the doors to support almost
everything as a host package. But maybe there's nothing wrong with that either.

 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 2/2] jq: enable host package

[Thomas Petazzoni]

Arnout,

On Wed, 21 Oct 2015 23:04:21 +0200, Arnout Vandecappelle wrote:

>  I was thinking the same thing when I say the patch, but I don't care enough to
> really comment on it. It's a bit similar to packages that depend on some host
> preprocessing tool, like flex. But then of course we just have a dependency and
> not a Config.in.host.

To me, host packages that are a dependency of some other target
packages have a real and strong reason to exist. The ones that don't
should really fall into the "useful for debugging, flashing or
preparing images" category IMO.

>  OTOH, if we accept this, it kind of opens the doors to support almost
> everything as a host package. But maybe there's nothing wrong with that either.

Opening the door to almost everything as a host package is personally
something that I'd like to avoid.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2] jq: enable host package

[Thomas De Schampheleire]

On Wed, Oct 21, 2015 at 11:20 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Arnout,
>
> On Wed, 21 Oct 2015 23:04:21 +0200, Arnout Vandecappelle wrote:
>
>>  I was thinking the same thing when I say the patch, but I don't care enough to
>> really comment on it. It's a bit similar to packages that depend on some host
>> preprocessing tool, like flex. But then of course we just have a dependency and
>> not a Config.in.host.
>
> To me, host packages that are a dependency of some other target
> packages have a real and strong reason to exist. The ones that don't
> should really fall into the "useful for debugging, flashing or
> preparing images" category IMO.
>
>>  OTOH, if we accept this, it kind of opens the doors to support almost
>> everything as a host package. But maybe there's nothing wrong with that either.
>
> Opening the door to almost everything as a host package is personally
> something that I'd like to avoid.

I don't think that we would open the door to 'almost every package' as
host package. For many packages, it makes no sense to have them as
host package in the context of buildroot, say audio/video,
profiling/benchmark, hardware handling, mail, miscellaneous,
networking apps, ...
In fact, I would argue that almost none of the packages we have in
Buildroot make sense as host packages, except those that we already
have, and a few limited categories of tools. In my opinion, one such
category could be config file handling, and host-jq fits into that
category. It's the type of thing one can reasonably need to put
together filestystem images for embedded devices, especially in
environments where many (variants of) devices are being developed.

/Thomas

[Buildroot] [PATCH 2/2] jq: enable host package

[Arnout Vandecappelle]

On 22-10-15 10:22, Thomas De Schampheleire wrote:
> On Wed, Oct 21, 2015 at 11:20 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Arnout,
>>
>> On Wed, 21 Oct 2015 23:04:21 +0200, Arnout Vandecappelle wrote:
>>
>>>  I was thinking the same thing when I say the patch, but I don't care enough to
>>> really comment on it. It's a bit similar to packages that depend on some host
>>> preprocessing tool, like flex. But then of course we just have a dependency and
>>> not a Config.in.host.
>>
>> To me, host packages that are a dependency of some other target
>> packages have a real and strong reason to exist. The ones that don't
>> should really fall into the "useful for debugging, flashing or
>> preparing images" category IMO.
>>
>>>  OTOH, if we accept this, it kind of opens the doors to support almost
>>> everything as a host package. But maybe there's nothing wrong with that either.
>>
>> Opening the door to almost everything as a host package is personally
>> something that I'd like to avoid.
> 
> I don't think that we would open the door to 'almost every package' as
> host package. For many packages, it makes no sense to have them as
> host package in the context of buildroot, say audio/video,
> profiling/benchmark, hardware handling, mail, miscellaneous,
> networking apps, ...
> In fact, I would argue that almost none of the packages we have in
> Buildroot make sense as host packages, except those that we already
> have, and a few limited categories of tools. In my opinion, one such
> category could be config file handling, and host-jq fits into that
> category. It's the type of thing one can reasonably need to put
> together filestystem images for embedded devices, especially in
> environments where many (variants of) devices are being developed.

 So that basically puts jq into the 'image generation' category for you :-)

 Note that personally I don't have much of a problem with adding more host
packages, I don't think they tend to add much of a maintenance burden.

 Regards,
 Arnout



-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 2/2] jq: enable host package

[Yann E. MORIN]

Thomas?, All,

On 2015-10-21 22:39 +0200, Thomas Petazzoni spake thusly:
> On Mon, 19 Oct 2015 12:42:09 +0200, Thomas De Schampheleire wrote:
> > From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> > 
> > Allow building jq as host utility for use in post-build scripts.
> > This can be useful to created, edit, merge or even perform syntax checking
> > on JSON files.
> > 
> > Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> Since this host package clearly doesn't fall into the "debugging tool,
> flashing tool or image preparation tool" category, I'd like to collect
> the opinion of a few other developers before applying.
> 
> Peter, Arnout, Yann, what do you think?

On principle, I'm OK with user-selectable host packages, as long as there
is a real benefit to having it, like:
  - image preparation tools (fs, flash, dtc, bootloader utils...)
  - configuration tools (pwgen, SElinux compiler...)
  - debug tools

(Note: I'm not listing dependency of target package, since that is not
user-selectable.)

Adding any other type of package to be user-selectable will always be
debatable. I'm not sure where we should draw the line, but we could
provide guidelines, like:

  - the package is not usually packaged in distributions (or too old);
    libxslt for xsltproc comes to mind for example;

  - the submitter comes with a very clear use-case which can not
    otherwise be solved by other means (overlays, conditionals in
    post-build scripts...)

Now, as far as host-jq is concerned, I'm not very sure. The use-case
Thomas DS. invoques is tweaking config files for various targets.

Even though I understand that, and even though I could have thought the
same, I'm still not quite sure. JSON is indeed a pretty much usual
configuration format those days (whether that is sane is a question for
another thread ;-] ). However, I would argue (weakly) that this tool
should be provided by the host environment, not us.

Except: old, enterprise-class distributions (e.g. RHEL5) may lack that
tool, especially since it was born in 2012.

So, really, I wouln't really argue either way.

But surely, this would be our first host-package that is not in the
categories I listed above.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/2] jq: enable host package

[Thomas De Schampheleire]

On Thu, Oct 22, 2015 at 6:39 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Thomas?, All,
>
> On 2015-10-21 22:39 +0200, Thomas Petazzoni spake thusly:
>> On Mon, 19 Oct 2015 12:42:09 +0200, Thomas De Schampheleire wrote:
>> > From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>> >
>> > Allow building jq as host utility for use in post-build scripts.
>> > This can be useful to created, edit, merge or even perform syntax checking
>> > on JSON files.
>> >
>> > Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
>>
>> Since this host package clearly doesn't fall into the "debugging tool,
>> flashing tool or image preparation tool" category, I'd like to collect
>> the opinion of a few other developers before applying.
>>
>> Peter, Arnout, Yann, what do you think?
>
> On principle, I'm OK with user-selectable host packages, as long as there
> is a real benefit to having it, like:
>   - image preparation tools (fs, flash, dtc, bootloader utils...)
>   - configuration tools (pwgen, SElinux compiler...)
>   - debug tools
>
> (Note: I'm not listing dependency of target package, since that is not
> user-selectable.)
>
> Adding any other type of package to be user-selectable will always be
> debatable. I'm not sure where we should draw the line, but we could
> provide guidelines, like:
>
>   - the package is not usually packaged in distributions (or too old);
>     libxslt for xsltproc comes to mind for example;
>
>   - the submitter comes with a very clear use-case which can not
>     otherwise be solved by other means (overlays, conditionals in
>     post-build scripts...)
>
> Now, as far as host-jq is concerned, I'm not very sure. The use-case
> Thomas DS. invoques is tweaking config files for various targets.
>
> Even though I understand that, and even though I could have thought the
> same, I'm still not quite sure. JSON is indeed a pretty much usual
> configuration format those days (whether that is sane is a question for
> another thread ;-] ). However, I would argue (weakly) that this tool
> should be provided by the host environment, not us.
>
> Except: old, enterprise-class distributions (e.g. RHEL5) may lack that
> tool, especially since it was born in 2012.

Even for RHEL6, while there possibly is an rpm package available,
tools as jq are not installed by default.
For products like those you build with buildroot, the build should be
self-contained and not depend too much on environment tools. For
things like grep, (host-)gcc, ... we can reasonably expect them to be
present. But I don't see jq in this category. At least in our case,
with all workstations involved in the organization, it is not possible
to expect all of them to contain jq.

In practice, we are using JSON files to pass some configuration to
applications. Different boards need different configuration, and jq
provides easy manipulation and reading of such files on the cmd-line
(and jansson can be used to read such files from the application).
Some of our JSON files are pre-existing in a rootfs overlay; in this
case we use host-jq to run a syntax check.
Some of them are dynamically generated and combined from the build
process using host-jq.

Hope this helps to convice you all :)

/Thomas

[Buildroot] [PATCH 2/2] jq: enable host package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas De Schampheleire <patrickdepinguin@gmail.com> writes:

Hi,

 > On Thu, Oct 22, 2015 at 6:39 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:

 > In practice, we are using JSON files to pass some configuration to
 > applications. Different boards need different configuration, and jq
 > provides easy manipulation and reading of such files on the cmd-line
 > (and jansson can be used to read such files from the application).
 > Some of our JSON files are pre-existing in a rootfs overlay; in this
 > case we use host-jq to run a syntax check.
 > Some of them are dynamically generated and combined from the build
 > process using host-jq.

 > Hope this helps to convice you all :)

For me it does atleast - Committed, thanks.

-- 
Venlig hilsen,
Peter Korsgaard 

[Buildroot] [PATCH 1/2] jq: disable maintainer mode

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Mon, 19 Oct 2015 12:42:08 +0200, Thomas De Schampheleire wrote:
> From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
> 
> Since version 1.5, maintainer mode is enabled by default. We don't need
> this, and is in particular problematic because it requires bison 3.0.
> 
> Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

It took me a little bit of time to understand this one, since jq 1.5
was building fine in my minimal chroot that does not have bison
installed.

However, what happens is that if you don't have bison installed at all,
everything works fine. But if you have a bison version installed that
happens to be older than bison 3.0, then you get an error at configure
time.

So: patch applied, thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com