[Buildroot] [PATCH] gpsd: fix Python-related build failure

[Buildroot] [PATCH] gpsd: fix Python-related build failure

[spdawson at gmail.com]

From: Simon Dawson <spdawson@gmail.com>

The gpsd build falls over on certain autobuild machines; an example follows.

  http://autobuild.buildroot.net/results/42b435c271b0d791365e18ad974c7eecca8896a0/build-end.log

ImportError: No module named simplejson:
  File "/scratch/peko/build/gpsd-3.7/SConstruct", line 1072:
    from leapsecond import save_leapseconds
  File "/scratch/peko/build/gpsd-3.7/leapsecond.py", line 27:
    import gps.misc
  File "/scratch/peko/build/gpsd-3.7/gps/__init__.py", line 9:
    from gps import *
  File "/scratch/peko/build/gpsd-3.7/gps/gps.py", line 17:
    from client import *
  File "/scratch/peko/build/gpsd-3.7/gps/client.py", line 9:
    import simplejson as json                        # For Python 2.4 and 2.5
make: *** [/scratch/peko/build/gpsd-3.7/.stamp_built] Error 2

The problem appears to be the indiscriminate importing done in the module
initialisation for the gps Python module. If the simplejson module is not
available for the host Python, then the build fails.

For the purposes of the build, the simplejson import is superfluous; in fact,
since SConstruct pulls in gps.misc via leapsecond.py, all of the imports in
the gps module initialisation are superfluous.

Signed-off-by: Simon Dawson <spdawson@gmail.com>
---
 .../gpsd-05-python-2.5-compat-simplejson.patch     |   40 ++++++++++++++++++++
 1 file changed, 40 insertions(+)
 create mode 100644 package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch

diff --git a/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch b/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch
new file mode 100644
index 0000000..f275faa
--- /dev/null
+++ b/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch
@@ -0,0 +1,40 @@
+The gpsd build falls over on certain autobuild machines; an example follows.
+
+  http://autobuild.buildroot.net/results/42b435c271b0d791365e18ad974c7eecca8896a0/build-end.log
+
+ImportError: No module named simplejson:
+  File "/scratch/peko/build/gpsd-3.7/SConstruct", line 1072:
+    from leapsecond import save_leapseconds
+  File "/scratch/peko/build/gpsd-3.7/leapsecond.py", line 27:
+    import gps.misc
+  File "/scratch/peko/build/gpsd-3.7/gps/__init__.py", line 9:
+    from gps import *
+  File "/scratch/peko/build/gpsd-3.7/gps/gps.py", line 17:
+    from client import *
+  File "/scratch/peko/build/gpsd-3.7/gps/client.py", line 9:
+    import simplejson as json                        # For Python 2.4 and 2.5
+make: *** [/scratch/peko/build/gpsd-3.7/.stamp_built] Error 2
+
+The problem appears to be the indiscriminate importing done in the module
+initialisation for the gps Python module. If the simplejson module is not
+available for the host Python, then the build fails.
+
+For the purposes of the build, the simplejson import is superfluous; in fact,
+since SConstruct pulls in gps.misc via leapsecond.py, all of the imports in
+the gps module initialisation are superfluous.
+
+Signed-off-by: Simon Dawson <spdawson@gmail.com>
+diff -Nurp a/gps/__init__.py b/gps/__init__.py
+--- a/gps/__init__.py	2012-05-23 22:06:40.000000000 +0100
++++ b/gps/__init__.py	2012-08-03 09:09:54.096816764 +0100
+@@ -6,8 +6,8 @@
+ api_major_version = 5   # bumped on incompatible changes
+ api_minor_version = 0   # bumped on compatible changes
+ 
+-from gps import *
+-from misc import *
++#from gps import *
++#from misc import *
+ 
+ # The 'client' module exposes some C utility functions for Python clients.
+ # The 'packet' module exposes the packet getter via a Python interface.
-- 
1.7.9.5

[Buildroot] [PATCH] gpsd: fix Python-related build failure

[Maxime Ripard]

Hi,

Le 21/08/2012 13:31, spdawson at gmail.com a ?crit :
> From: Simon Dawson <spdawson@gmail.com>
> 
> The gpsd build falls over on certain autobuild machines; an example follows.
> 
>   http://autobuild.buildroot.net/results/42b435c271b0d791365e18ad974c7eecca8896a0/build-end.log
> 
> ImportError: No module named simplejson:
>   File "/scratch/peko/build/gpsd-3.7/SConstruct", line 1072:
>     from leapsecond import save_leapseconds
>   File "/scratch/peko/build/gpsd-3.7/leapsecond.py", line 27:
>     import gps.misc
>   File "/scratch/peko/build/gpsd-3.7/gps/__init__.py", line 9:
>     from gps import *
>   File "/scratch/peko/build/gpsd-3.7/gps/gps.py", line 17:
>     from client import *
>   File "/scratch/peko/build/gpsd-3.7/gps/client.py", line 9:
>     import simplejson as json                        # For Python 2.4 and 2.5
> make: *** [/scratch/peko/build/gpsd-3.7/.stamp_built] Error 2
> 
> The problem appears to be the indiscriminate importing done in the module
> initialisation for the gps Python module. If the simplejson module is not
> available for the host Python, then the build fails.
> 
> For the purposes of the build, the simplejson import is superfluous; in fact,
> since SConstruct pulls in gps.misc via leapsecond.py, all of the imports in
> the gps module initialisation are superfluous.
> 
> Signed-off-by: Simon Dawson <spdawson@gmail.com>
> ---
>  .../gpsd-05-python-2.5-compat-simplejson.patch     |   40 ++++++++++++++++++++
>  1 file changed, 40 insertions(+)
>  create mode 100644 package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch
> 
> diff --git a/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch b/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch
> new file mode 100644
> index 0000000..f275faa
> --- /dev/null
> +++ b/package/gpsd/gpsd-05-python-2.5-compat-simplejson.patch
> @@ -0,0 +1,40 @@
> +The gpsd build falls over on certain autobuild machines; an example follows.
> +
> +  http://autobuild.buildroot.net/results/42b435c271b0d791365e18ad974c7eecca8896a0/build-end.log
> +
> +ImportError: No module named simplejson:
> +  File "/scratch/peko/build/gpsd-3.7/SConstruct", line 1072:
> +    from leapsecond import save_leapseconds
> +  File "/scratch/peko/build/gpsd-3.7/leapsecond.py", line 27:
> +    import gps.misc
> +  File "/scratch/peko/build/gpsd-3.7/gps/__init__.py", line 9:
> +    from gps import *
> +  File "/scratch/peko/build/gpsd-3.7/gps/gps.py", line 17:
> +    from client import *
> +  File "/scratch/peko/build/gpsd-3.7/gps/client.py", line 9:
> +    import simplejson as json                        # For Python 2.4 and 2.5
> +make: *** [/scratch/peko/build/gpsd-3.7/.stamp_built] Error 2
> +
> +The problem appears to be the indiscriminate importing done in the module
> +initialisation for the gps Python module. If the simplejson module is not
> +available for the host Python, then the build fails.
> +
> +For the purposes of the build, the simplejson import is superfluous; in fact,
> +since SConstruct pulls in gps.misc via leapsecond.py, all of the imports in
> +the gps module initialisation are superfluous.
> +
> +Signed-off-by: Simon Dawson <spdawson@gmail.com>
> +diff -Nurp a/gps/__init__.py b/gps/__init__.py
> +--- a/gps/__init__.py	2012-05-23 22:06:40.000000000 +0100
> ++++ b/gps/__init__.py	2012-08-03 09:09:54.096816764 +0100
> +@@ -6,8 +6,8 @@
> + api_major_version = 5   # bumped on incompatible changes
> + api_minor_version = 0   # bumped on compatible changes
> + 
> +-from gps import *
> +-from misc import *
> ++#from gps import *
> ++#from misc import *
> + 
> + # The 'client' module exposes some C utility functions for Python clients.
> + # The 'packet' module exposes the packet getter via a Python interface.
> 

I'm not very fond of that approach. I guess one of the purpose of these
bindings are precisely to provide a python API, and you break it here
since instead of including gps.foo, you will need to include gps.bar.foo
(if I remember my Python correctly).

I guess the real problem that we have to address here is why scons uses
the interpreter from the distribution and not the buildroot-generated one.

I guess, we could just make scons depends on host-python and force the
$(SCONS) variable as "$(HOST_DIR)/usr/bin/python
$(HOST_DIR)/usr/bin/scons ?"

-- 
Maxime Ripard, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] gpsd: fix Python-related build failure

[Simon Dawson]

Hi Maxime; thanks for the comments.

On 22 August 2012 09:07, Maxime Ripard <maxime.ripard@free-electrons.com> wrote:
> I'm not very fond of that approach. I guess one of the purpose of these
> bindings are precisely to provide a python API, and you break it here
> since instead of including gps.foo, you will need to include gps.bar.foo
> (if I remember my Python correctly).

Yes, agreed; it does break the Python bindings for gpsd. This might
not be such an issue, however, as these bindings are disabled in the
Buildroot gpsd package configuration at present.

> I guess the real problem that we have to address here is why scons uses
> the interpreter from the distribution and not the buildroot-generated one.
>
> I guess, we could just make scons depends on host-python and force the
> $(SCONS) variable as "$(HOST_DIR)/usr/bin/python
> $(HOST_DIR)/usr/bin/scons ?"

This sounds to me like a good idea. I had tried to avoid the
host-python dependency, which was in fact present in an earlier
incarnation of the gpsd version bump patch. But it looks like it might
be the cleanest way of resolving the present issue.

Simon.

[Buildroot] [PATCH] gpsd: fix Python-related build failure

[Thomas Petazzoni]

Le Wed, 22 Aug 2012 09:26:06 +0100,
Simon Dawson <spdawson@gmail.com> a ?crit :

> > I guess the real problem that we have to address here is why scons uses
> > the interpreter from the distribution and not the buildroot-generated one.
> >
> > I guess, we could just make scons depends on host-python and force the
> > $(SCONS) variable as "$(HOST_DIR)/usr/bin/python
> > $(HOST_DIR)/usr/bin/scons ?"
> 
> This sounds to me like a good idea. I had tried to avoid the
> host-python dependency, which was in fact present in an earlier
> incarnation of the gpsd version bump patch. But it looks like it might
> be the cleanest way of resolving the present issue.

If we could avoid the host-python dependency, it would be great. My
understanding is that the SConstruct needs JSON just to generate one
file. Maybe we could simply integrate a pre-generated version as a
patch. But maybe it's too hackish, and we should just go ahead and add
host-python as a dependency of scons, so that a well-known version of
Python gets built.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] gpsd: Fix lacking simplejson module error at build

[Maxime Ripard]

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
---

Thomas,

Would something like this make more sense to you ?

Maxime

 .../gpsd-05-fix-leapsecond-script-python2.5.patch  |   71 ++++++++++++++++++++
 1 file changed, 71 insertions(+)
 create mode 100644 package/gpsd/gpsd-05-fix-leapsecond-script-python2.5.patch

diff --git a/package/gpsd/gpsd-05-fix-leapsecond-script-python2.5.patch b/package/gpsd/gpsd-05-fix-leapsecond-script-python2.5.patch
new file mode 100644
index 0000000..8866b81
--- /dev/null
+++ b/package/gpsd/gpsd-05-fix-leapsecond-script-python2.5.patch
@@ -0,0 +1,71 @@
+the json module was added with python2.6, so a regular python 2.5
+machine will lack this module and won't probably have the simplejson
+module imported by the leapsecond.py script.
+
+Since the only function used is the isotime function, which is
+self-contained and quite trivial, only copy this function into the
+leapsecond script to avoid the import of the gps.misc module, which
+needs simplejson.
+
+Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
+---
+ leapsecond.py |   27 ++++++++++++++++++++++++---
+ 1 file changed, 24 insertions(+), 3 deletions(-)
+
+diff --git a/leapsecond.py b/leapsecond.py
+index 2059f6c..cdacdb4 100755
+--- a/leapsecond.py
++++ b/leapsecond.py
+@@ -24,7 +24,6 @@
+ # BSD terms apply: see the file COPYING in the distribution root for details.
+ #
+ import os, urllib, re, random, time, calendar, math, sys
+-import gps.misc
+ 
+ __locations = [
+     (
+@@ -48,6 +47,28 @@ __locations = [
+ # between times it might change, in seconds since Unix epoch GMT.
+ __cachepath = "/var/run/leapsecond"
+ 
++def isotime(s):
++    "Convert timestamps in ISO8661 format to and from Unix time."
++    if type(s) == type(1):
++        return time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(s))
++    elif type(s) == type(1.0):
++        date = int(s)
++        msec = s - date
++        date = time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(s))
++        return date + "." + repr(msec)[3:]
++    elif type(s) == type("") or type(s) == type(u""):
++        if s[-1] == "Z":
++            s = s[:-1]
++        if "." in s:
++            (date, msec) = s.split(".")
++        else:
++            date = s
++            msec = "0"
++        # Note: no leap-second correction!
++        return calendar.timegm(time.strptime(date, "%Y-%m-%dT%H:%M:%S")) + float("0." + msec)
++    else:
++        raise TypeError
++
+ def retrieve():
+     "Retrieve current leap-second from Web sources."
+     random.shuffle(__locations)	# To spread the load
+@@ -261,10 +282,10 @@ if __name__ == '__main__':
+             print unix_to_rfc822(float(val))
+             raise SystemExit, 0
+         elif (switch == '-I'):  # Compute Unix time from ISO8601 date
+-            print gps.misc.isotime(val)
++            print isotime(val)
+             raise SystemExit, 0
+         elif (switch == '-O'):  # Compute ISO8601 date from Unix time
+-            print gps.misc.isotime(float(val))
++            print isotime(float(val))
+             raise SystemExit, 0
+ 
+         print "Current leap second:", retrieve()
+-- 
+1.7.9.5
+
-- 
1.7.9.5

[Buildroot] [PATCH] gpsd: Fix lacking simplejson module error at build

[Thomas Petazzoni]

Le Fri, 24 Aug 2012 09:59:29 +0200,
Maxime Ripard <maxime.ripard@free-electrons.com> a ?crit :

> Would something like this make more sense to you ?

Yes, this looks good. Long term, we will just deprecate the support for
Python 2.5 as the host Python, but in the mean time, this looks like
the easiest fix.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] gpsd: Fix lacking simplejson module error at build

[Thomas Petazzoni]

Le Fri, 24 Aug 2012 09:59:29 +0200,
Maxime Ripard <maxime.ripard@free-electrons.com> a ?crit :

> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com