[PATCH] pango: use qemu to generate pango.modules during rootfs construction

[PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

This allows pango to be installed in an image with read-only-rootfs.

Signed-off-by: Phil Blundell <philb@gnu.org>
---
 meta/recipes-graphics/pango/pango.inc       |   11 +++++++----
 meta/recipes-graphics/pango/pango_1.28.4.bb |    2 +-
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-graphics/pango/pango.inc b/meta/recipes-graphics/pango/pango.inc
index a8e99e2..ff3f5dd 100644
--- a/meta/recipes-graphics/pango/pango.inc
+++ b/meta/recipes-graphics/pango/pango.inc
@@ -10,13 +10,13 @@ SECTION = "libs"
 LICENSE = "LGPL"
 
 X11DEPENDS = "virtual/libx11 libxft"
-DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
+DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo qemu-native ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
 
 PACKAGES_DYNAMIC = "pango-module-*"
 
 RRECOMMENDS_${PN} = "${@base_contains('DISTRO_FEATURES', 'x11', 'pango-module-basic-x', '', d)} pango-module-basic-fc" 
 
-inherit gnome
+inherit gnome qemu
 
 EXTRA_AUTORECONF = ""
 
@@ -33,9 +33,12 @@ LIBV = "1.6.0"
 
 postinst_prologue() {
 if [ "x$D" != "x" ]; then
-  exit 1
+	if [ ! -f "$D${sysconfdir}/pango/pango.modules" ]; then
+		mkdir -p $D${sysconfdir}/pango
+		PSEUDO_RELOADED=YES ${@qemu_target_binary(d)} -L $D $D${bindir}/pango-querymodules > "$D${sysconfdir}/pango/pango.modules"
+	fi
+	exit 0
 fi
-
 }
 
 python populate_packages_prepend () {
diff --git a/meta/recipes-graphics/pango/pango_1.28.4.bb b/meta/recipes-graphics/pango/pango_1.28.4.bb
index 68ab700..33eac21 100644
--- a/meta/recipes-graphics/pango/pango_1.28.4.bb
+++ b/meta/recipes-graphics/pango/pango_1.28.4.bb
@@ -2,7 +2,7 @@ require pango.inc
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=3bf50002aefd002f49e7bb854063f7e7"
 
-PR = "r1"
+PR = "r4"
 
 SRC_URI += "file://no-tests.patch"
 
-- 
1.7.4.1

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Saul Wold]

On 08/31/2011 08:35 AM, Phil Blundell wrote:
> This allows pango to be installed in an image with read-only-rootfs.
>
> Signed-off-by: Phil Blundell<philb@gnu.org>
> ---
>   meta/recipes-graphics/pango/pango.inc       |   11 +++++++----
>   meta/recipes-graphics/pango/pango_1.28.4.bb |    2 +-
>   2 files changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/meta/recipes-graphics/pango/pango.inc b/meta/recipes-graphics/pango/pango.inc
> index a8e99e2..ff3f5dd 100644
> --- a/meta/recipes-graphics/pango/pango.inc
> +++ b/meta/recipes-graphics/pango/pango.inc
> @@ -10,13 +10,13 @@ SECTION = "libs"
>   LICENSE = "LGPL"
>
>   X11DEPENDS = "virtual/libx11 libxft"
> -DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
> +DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo qemu-native ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
>
>   PACKAGES_DYNAMIC = "pango-module-*"
>
>   RRECOMMENDS_${PN} = "${@base_contains('DISTRO_FEATURES', 'x11', 'pango-module-basic-x', '', d)} pango-module-basic-fc"
>
> -inherit gnome
> +inherit gnome qemu
>
>   EXTRA_AUTORECONF = ""
>
> @@ -33,9 +33,12 @@ LIBV = "1.6.0"
>
>   postinst_prologue() {
>   if [ "x$D" != "x" ]; then
> -  exit 1
> +	if [ ! -f "$D${sysconfdir}/pango/pango.modules" ]; then
> +		mkdir -p $D${sysconfdir}/pango
> +		PSEUDO_RELOADED=YES ${@qemu_target_binary(d)} -L $D $D${bindir}/pango-querymodules>  "$D${sysconfdir}/pango/pango.modules"
> +	fi
> +	exit 0
>   fi
> -
>   }
>
>   python populate_packages_prepend () {
> diff --git a/meta/recipes-graphics/pango/pango_1.28.4.bb b/meta/recipes-graphics/pango/pango_1.28.4.bb
> index 68ab700..33eac21 100644
> --- a/meta/recipes-graphics/pango/pango_1.28.4.bb
> +++ b/meta/recipes-graphics/pango/pango_1.28.4.bb
> @@ -2,7 +2,7 @@ require pango.inc
>
>   LIC_FILES_CHKSUM = "file://COPYING;md5=3bf50002aefd002f49e7bb854063f7e7"
>
> -PR = "r1"
> +PR = "r4"
>
>   SRC_URI += "file://no-tests.patch"
>

Merged but we found a problem with qemux86-64, a bug was filed and you
should have received an email

Thanks
	Sau!

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Saul Wold]

On 08/31/2011 08:35 AM, Phil Blundell wrote:
> This allows pango to be installed in an image with read-only-rootfs.
>
> Signed-off-by: Phil Blundell<philb@gnu.org>
> ---
>   meta/recipes-graphics/pango/pango.inc       |   11 +++++++----
>   meta/recipes-graphics/pango/pango_1.28.4.bb |    2 +-
>   2 files changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/meta/recipes-graphics/pango/pango.inc b/meta/recipes-graphics/pango/pango.inc
> index a8e99e2..ff3f5dd 100644
> --- a/meta/recipes-graphics/pango/pango.inc
> +++ b/meta/recipes-graphics/pango/pango.inc
> @@ -10,13 +10,13 @@ SECTION = "libs"
>   LICENSE = "LGPL"
>
>   X11DEPENDS = "virtual/libx11 libxft"
> -DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
> +DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo qemu-native ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
>
>   PACKAGES_DYNAMIC = "pango-module-*"
>
>   RRECOMMENDS_${PN} = "${@base_contains('DISTRO_FEATURES', 'x11', 'pango-module-basic-x', '', d)} pango-module-basic-fc"
>
> -inherit gnome
> +inherit gnome qemu
>
>   EXTRA_AUTORECONF = ""
>
> @@ -33,9 +33,12 @@ LIBV = "1.6.0"
>
>   postinst_prologue() {
>   if [ "x$D" != "x" ]; then
> -  exit 1
> +	if [ ! -f "$D${sysconfdir}/pango/pango.modules" ]; then
> +		mkdir -p $D${sysconfdir}/pango
> +		PSEUDO_RELOADED=YES ${@qemu_target_binary(d)} -L $D $D${bindir}/pango-querymodules>  "$D${sysconfdir}/pango/pango.modules"
> +	fi
> +	exit 0
>   fi
> -
>   }
>
The usage here of qemu, and some further testing has shown that this may 
not be the best time to put this patch in, There are failures in both 
x86-64 and arm builds and the newer version of qemu does not appear to 
resolve this.

Phil, given we are trying to generate a release at this point, I feel 
that we need to revert this patch. You are welcome to resubmit an 
updated version with sufficient test results, but it will be to master 
and not 1.1.

Thanks for the efforts to create a read-only rootfs, this one just can't 
make it this time.

Sau!

>   python populate_packages_prepend () {
> diff --git a/meta/recipes-graphics/pango/pango_1.28.4.bb b/meta/recipes-graphics/pango/pango_1.28.4.bb
> index 68ab700..33eac21 100644
> --- a/meta/recipes-graphics/pango/pango_1.28.4.bb
> +++ b/meta/recipes-graphics/pango/pango_1.28.4.bb
> @@ -2,7 +2,7 @@ require pango.inc
>
>   LIC_FILES_CHKSUM = "file://COPYING;md5=3bf50002aefd002f49e7bb854063f7e7"
>
> -PR = "r1"
> +PR = "r4"
>
>   SRC_URI += "file://no-tests.patch"
>

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Richard Purdie]

On Fri, 2011-09-09 at 14:19 -0700, Saul Wold wrote:
> On 08/31/2011 08:35 AM, Phil Blundell wrote:
> > This allows pango to be installed in an image with read-only-rootfs.
> >
> > Signed-off-by: Phil Blundell<philb@gnu.org>
> > ---
> >   meta/recipes-graphics/pango/pango.inc       |   11 +++++++----
> >   meta/recipes-graphics/pango/pango_1.28.4.bb |    2 +-
> >   2 files changed, 8 insertions(+), 5 deletions(-)
> >
> > diff --git a/meta/recipes-graphics/pango/pango.inc b/meta/recipes-graphics/pango/pango.inc
> > index a8e99e2..ff3f5dd 100644
> > --- a/meta/recipes-graphics/pango/pango.inc
> > +++ b/meta/recipes-graphics/pango/pango.inc
> > @@ -10,13 +10,13 @@ SECTION = "libs"
> >   LICENSE = "LGPL"
> >
> >   X11DEPENDS = "virtual/libx11 libxft"
> > -DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
> > +DEPENDS = "glib-2.0 fontconfig freetype zlib virtual/libiconv gtk-doc-native cairo qemu-native ${@base_contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d)}"
> >
> >   PACKAGES_DYNAMIC = "pango-module-*"
> >
> >   RRECOMMENDS_${PN} = "${@base_contains('DISTRO_FEATURES', 'x11', 'pango-module-basic-x', '', d)} pango-module-basic-fc"
> >
> > -inherit gnome
> > +inherit gnome qemu
> >
> >   EXTRA_AUTORECONF = ""
> >
> > @@ -33,9 +33,12 @@ LIBV = "1.6.0"
> >
> >   postinst_prologue() {
> >   if [ "x$D" != "x" ]; then
> > -  exit 1
> > +	if [ ! -f "$D${sysconfdir}/pango/pango.modules" ]; then
> > +		mkdir -p $D${sysconfdir}/pango
> > +		PSEUDO_RELOADED=YES ${@qemu_target_binary(d)} -L $D $D${bindir}/pango-querymodules>  "$D${sysconfdir}/pango/pango.modules"
> > +	fi
> > +	exit 0
> >   fi
> > -
> >   }
> >
> The usage here of qemu, and some further testing has shown that this may 
> not be the best time to put this patch in, There are failures in both 
> x86-64 and arm builds and the newer version of qemu does not appear to 
> resolve this.
> 
> Phil, given we are trying to generate a release at this point, I feel 
> that we need to revert this patch. You are welcome to resubmit an 
> updated version with sufficient test results, but it will be to master 
> and not 1.1.
> 
> Thanks for the efforts to create a read-only rootfs, this one just can't 
> make it this time.

For what its worth, even on plain qemux86 I'm seeing:

qemu: Unsupported syscall: 240
inb: port=0x0000
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
/media/data1/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato-1.0-r0/rootfs/etc/rpm-postinsts/pango-module-basic-fc.sh: line 8: 11084 Segmentation fault      PSEUDO_RELOADED=YES qemu-i386 -L $D $D/usr/bin/pango-querymodules > "$D/etc/pango/pango.modules"

in the rootfs logs. Which reading things online suggests that qemu
doesn't support multithreading under qemu user emulation on i386.

I therefore think we're going to have to rethink this patch...

Cheers,

Richard

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

On Sat, 2011-09-10 at 09:24 +0100, Richard Purdie wrote:
> For what its worth, even on plain qemux86 I'm seeing:
> 
> qemu: Unsupported syscall: 240
> inb: port=0x0000
> qemu: uncaught target signal 11 (Segmentation fault) - core dumped
> /media/data1/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato-1.0-r0/rootfs/etc/rpm-postinsts/pango-module-basic-fc.sh: line 8: 11084 Segmentation fault      PSEUDO_RELOADED=YES qemu-i386 -L $D $D/usr/bin/pango-querymodules > "$D/etc/pango/pango.modules"
> 
> in the rootfs logs. Which reading things online suggests that qemu
> doesn't support multithreading under qemu user emulation on i386.

That is rather strange.  It does work ok for me on mips, and I wasn't
able to make it go wrong on x86-64 either in a quick test.

The other odd thing is that, from a quick look at the qemu source code,
it does seem as though sys_futex (which is syscall 240 on i386 and
syscall 202 on x86-64) ought to be supported if qemu was built with NPTL
on.  However, in my x86-64 test I did see the "unsupported syscall 202"
syscall although the binary still seemed to run fine.

And, well, the third weird thing is that your log message above implies
that pango-querymodules is trying to do inb(0) which clearly is not
going to work if you're not root.  I've no idea why it would be wanting
to do that.

p.

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Richard Purdie]

On Sat, 2011-09-10 at 10:28 +0100, Phil Blundell wrote:
> On Sat, 2011-09-10 at 09:24 +0100, Richard Purdie wrote:
> > For what its worth, even on plain qemux86 I'm seeing:
> > 
> > qemu: Unsupported syscall: 240
> > inb: port=0x0000
> > qemu: uncaught target signal 11 (Segmentation fault) - core dumped
> > /media/data1/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato-1.0-r0/rootfs/etc/rpm-postinsts/pango-module-basic-fc.sh: line 8: 11084 Segmentation fault      PSEUDO_RELOADED=YES qemu-i386 -L $D $D/usr/bin/pango-querymodules > "$D/etc/pango/pango.modules"
> > 
> > in the rootfs logs. Which reading things online suggests that qemu
> > doesn't support multithreading under qemu user emulation on i386.
> 
> That is rather strange.  It does work ok for me on mips, and I wasn't
> able to make it go wrong on x86-64 either in a quick test.
> 
> The other odd thing is that, from a quick look at the qemu source code,
> it does seem as though sys_futex (which is syscall 240 on i386 and
> syscall 202 on x86-64) ought to be supported if qemu was built with NPTL
> on.  However, in my x86-64 test I did see the "unsupported syscall 202"
> syscall although the binary still seemed to run fine.
> 
> And, well, the third weird thing is that your log message above implies
> that pango-querymodules is trying to do inb(0) which clearly is not
> going to work if you're not root.  I've no idea why it would be wanting
> to do that.

Its something about the context at rootfs generation time.

If I run the command manually, I see:

"""
sh-4.2$ /media/data1/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/qemu-i386 -L . ./usr/bin/pango-querymodules 
qemu: Unsupported syscall: 240
# Pango Modules file
# Automatically generated file, do not edit
#
qemu: Unsupported syscall: 240

(pango-querymodules:24343): GLib-WARNING **: getpwuid_r(): failed due to unknown user id (1000)
# ModulesPath = /usr/lib/pango/1.6.0/modules
#
qemu: Unsupported syscall: 240
/usr/lib/pango/1.6.0/modules/pango-basic-fc.so BasicScriptEngineFc PangoEngineShape PangoRenderFc latin:* cyrillic:* greek:* armenian:* georgian:* runic:* ogham:* bopomofo:* cherokee:* coptic:* deseret:* ethiopic:* gothic:* han:* hiragana:* katakana:* old-italic:* canadian-aboriginal:* yi:* braille:* cypriot:* limbu:* osmanya:* shavian:* linear-b:* ugaritic:* glagolitic:* cuneiform:* phoenician:* common:
/usr/lib/pango/1.6.0/modules/pango-basic-x.so BasicScriptEngineX PangoEngineShape PangoRenderX common:
"""

which looks right. It does crash as part of rpm rootfs generation
though, possibly something to do with pseudo given the root operation it
attempts. Did you test this against rpm or ipk?

Cheers,

Richard

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

On Sat, 2011-09-10 at 10:54 +0100, Richard Purdie wrote:
> which looks right. It does crash as part of rpm rootfs generation
> though, possibly something to do with pseudo given the root operation it
> attempts. Did you test this against rpm or ipk?

Ah, right, I was testing with ipk.  Maybe that's the difference.

p.

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

On Sat, 2011-09-10 at 10:28 +0100, Phil Blundell wrote: 
> The other odd thing is that, from a quick look at the qemu source code,
> it does seem as though sys_futex (which is syscall 240 on i386 and
> syscall 202 on x86-64) ought to be supported if qemu was built with NPTL
> on.  However, in my x86-64 test I did see the "unsupported syscall 202"
> syscall although the binary still seemed to run fine.

This part turns out not (quite) to be so mysterious.  For some reason,
qemu's configure script doesn't set target_nptl=yes for i386 or x86-64,
though it does for the majority of other arches.  So it will indeed be
the case that sys_futex will show up as an "unsupported syscall" on
those two architectures.

I'm not quite sure why qemu would be deciding not to do futexes on those
two architectures specifically.  I guess there must be some
functionality missing from the x86 emulation backend though I am not
sure exactly what that would be or how hard it would be to add.

p.

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Richard Purdie]

On Sat, 2011-09-10 at 12:53 +0100, Phil Blundell wrote:
> On Sat, 2011-09-10 at 10:54 +0100, Richard Purdie wrote:
> > which looks right. It does crash as part of rpm rootfs generation
> > though, possibly something to do with pseudo given the root operation it
> > attempts. Did you test this against rpm or ipk?
> 
> Ah, right, I was testing with ipk.  Maybe that's the difference.

I think the recent pseudo environment fixes which I ran into in other
areas have helped with this. Rather than a segfault, I now see:

 Adding system startup for /media/data1/build1/poky/build/tmp/work/qemux86-poky-linux/core-image-sato-1.0-r0/rootfs/etc/init.d/ofono.
qemu: Unsupported syscall: 240
qemu: Unsupported syscall: 240

(pango-querymodules:10498): GLib-WARNING **: getpwuid_r(): failed due to unknown user id (1000)
qemu: Unsupported syscall: 240

with the unsppported syscall issues being due to the lack of
multithreading support in qemu user mode emulation on x86. I suspect the
problem was that it was running in pseudo context when it shouldn't have
been and since it was root, it tried to do things as root which it
couldn't do.

Despite the above messages, it does now "work" on 32 bit x86 targets
however the warning message about the invalid user is yet another sign
of us needing to better think through this though more carefully (and
further test things) :/.

I've therefore taken the decision to revert this since I really want to
stablise for the release at this point and I'm not filled with
confidence that there aren't other problems lurking here.

Cheers,

Richard

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Otavio Salvador]

On Tue, Sep 13, 2011 at 14:08, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
...
> I've therefore taken the decision to revert this since I really want to
> stablise for the release at this point and I'm not filled with
> confidence that there aren't other problems lurking here.

With a close release this seems to be the right decision but I do like
the idea behind the patch and I think we ought to fix those for next
release.

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

On Tue, 2011-09-13 at 18:08 +0100, Richard Purdie wrote:
> I've therefore taken the decision to revert this since I really want to
> stablise for the release at this point and I'm not filled with
> confidence that there aren't other problems lurking here.

Okay, fair enough.  Actually I thought Saul had already reverted it a
few days ago but maybe I misunderstood what he was saying.

Do you know how long it's likely to be before this release is complete
and the tree is re-opened for development?  I'm a bit unclear what the
process is for that.

p.

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Otavio Salvador]

On Tue, Sep 13, 2011 at 14:25, Phil Blundell <philb@gnu.org> wrote:
> Do you know how long it's likely to be before this release is complete
> and the tree is re-opened for development?  I'm a bit unclear what the
> process is for that.

Maybe we ought to use a 'next' branch for it?

-- 
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Saul Wold]

On Tue, 2011-09-13 at 15:33 -0300, Otavio Salvador wrote:
> On Tue, Sep 13, 2011 at 14:25, Phil Blundell <philb@gnu.org> wrote:
> > Do you know how long it's likely to be before this release is complete
> > and the tree is re-opened for development?  I'm a bit unclear what the
> > process is for that.
> 
> Maybe we ought to use a 'next' branch for it?
> 
I believe the plan is to branch in the next day or so for an RC2 and
then master will open up for development again.  I will be pulling
together both RC2 pull requests (very limited point fixes) and look at
consolidated pulls for master.

Sau!

Re: [PATCH] pango: use qemu to generate pango.modules during rootfs construction

[Phil Blundell]

On Tue, 2011-09-13 at 15:33 -0300, Otavio Salvador wrote:
> On Tue, Sep 13, 2011 at 14:25, Phil Blundell <philb@gnu.org> wrote:
> > Do you know how long it's likely to be before this release is complete
> > and the tree is re-opened for development?  I'm a bit unclear what the
> > process is for that.
> 
> Maybe we ought to use a 'next' branch for it?

If master is going to be closed for a long time (weeks/months) then yes,
that seems like a good idea.  If we're only talking about a few days
then it's probably not worth the effort.

p.