[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

https://bugzilla.redhat.com/show_bug.cgi?id=1297855
Build log:
http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log

In brief the build fails at:

+ /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
Fatal error: exception Failure("dump: impossible tag (1002)")

When thinking about this bug, don't exclude the possibility that there
could be a problem in the OCaml compiler.  In Fedora we currently use
a non-upstream ppc64le code generator, although we are planning to
replace it soon(-ish) with the new upstream POWER code generator that
Xavier wrote last year.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Julia Lawall]

On Wed, 13 Jan 2016, Richard W.M. Jones wrote:

>
> https://bugzilla.redhat.com/show_bug.cgi?id=1297855
> Build log:
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log
>
> In brief the build fails at:
>
> + /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
> init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
> Fatal error: exception Failure("dump: impossible tag (1002)")
>
> When thinking about this bug, don't exclude the possibility that there
> could be a problem in the OCaml compiler.  In Fedora we currently use
> a non-upstream ppc64le code generator, although we are planning to
> replace it soon(-ish) with the new upstream POWER code generator that
> Xavier wrote last year.

I would suspect that there is some structure type that has been added to
OCaml that we don't take into account.  I'll try to find out more
information.

julia

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Julia Lawall]

On Wed, 13 Jan 2016, Richard W.M. Jones wrote:

>
> https://bugzilla.redhat.com/show_bug.cgi?id=1297855
> Build log:
> http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log
>
> In brief the build fails at:
>
> + /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
> init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
> Fatal error: exception Failure("dump: impossible tag (1002)")
>
> When thinking about this bug, don't exclude the possibility that there
> could be a problem in the OCaml compiler.  In Fedora we currently use
> a non-upstream ppc64le code generator, although we are planning to
> replace it soon(-ish) with the new upstream POWER code generator that
> Xavier wrote last year.

It is actually your code that is crashing :)

(* Dump an OCaml value into a printable string.
 * By Richard W.M. Jones (rich at annexia.org).
 * dumper.ml 1.2 2005/02/06 12:38:21 rich Exp
 *)

Maybe you could send the output of man Obj in your version of Ocaml?

julia

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

On Wed, Jan 13, 2016 at 03:17:36PM +0100, Julia Lawall wrote:
> 
> 
> On Wed, 13 Jan 2016, Richard W.M. Jones wrote:
> 
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1297855
> > Build log:
> > http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log
> >
> > In brief the build fails at:
> >
> > + /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
> > init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
> > Fatal error: exception Failure("dump: impossible tag (1002)")
> >
> > When thinking about this bug, don't exclude the possibility that there
> > could be a problem in the OCaml compiler.  In Fedora we currently use
> > a non-upstream ppc64le code generator, although we are planning to
> > replace it soon(-ish) with the new upstream POWER code generator that
> > Xavier wrote last year.
> 
> It is actually your code that is crashing :)
> 
> (* Dump an OCaml value into a printable string.
>  * By Richard W.M. Jones (rich at annexia.org).
>  * dumper.ml 1.2 2005/02/06 12:38:21 rich Exp
>  *)
> 
> Maybe you could send the output of man Obj in your version of Ocaml?

Not sure I understand - we don't change the Obj man page at all.

The implementation of OCaml objects (as in the object header and so
on) should also be the same, AFAIK.  Certainly the ppc64le backend
doesn't make any changes inside the OCaml compiler core, eg. to the GC
or libraries.  It's a 64 bit LE platform, so AFAIK the memory
structures ought to be identical to x86-64.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Julia Lawall]

On Wed, 13 Jan 2016, Richard W.M. Jones wrote:

> On Wed, Jan 13, 2016 at 03:17:36PM +0100, Julia Lawall wrote:
> >
> >
> > On Wed, 13 Jan 2016, Richard W.M. Jones wrote:
> >
> > >
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1297855
> > > Build log:
> > > http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log
> > >
> > > In brief the build fails at:
> > >
> > > + /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
> > > init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
> > > Fatal error: exception Failure("dump: impossible tag (1002)")
> > >
> > > When thinking about this bug, don't exclude the possibility that there
> > > could be a problem in the OCaml compiler.  In Fedora we currently use
> > > a non-upstream ppc64le code generator, although we are planning to
> > > replace it soon(-ish) with the new upstream POWER code generator that
> > > Xavier wrote last year.
> >
> > It is actually your code that is crashing :)
> >
> > (* Dump an OCaml value into a printable string.
> >  * By Richard W.M. Jones (rich at annexia.org).
> >  * dumper.ml 1.2 2005/02/06 12:38:21 rich Exp
> >  *)
> >
> > Maybe you could send the output of man Obj in your version of Ocaml?
>
> Not sure I understand - we don't change the Obj man page at all.
>
> The implementation of OCaml objects (as in the object header and so
> on) should also be the same, AFAIK.  Certainly the ppc64le backend
> doesn't make any changes inside the OCaml compiler core, eg. to the GC
> or libraries.  It's a 64 bit LE platform, so AFAIK the memory
> structures ought to be identical to x86-64.

Actually, I'm not sure to understand which version of OCaml you are using?

julia

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

On Wed, Jan 13, 2016 at 03:26:50PM +0100, Julia Lawall wrote:
> 
> 
> On Wed, 13 Jan 2016, Richard W.M. Jones wrote:
> 
> > On Wed, Jan 13, 2016 at 03:17:36PM +0100, Julia Lawall wrote:
> > >
> > >
> > > On Wed, 13 Jan 2016, Richard W.M. Jones wrote:
> > >
> > > >
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=1297855
> > > > Build log:
> > > > http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/345/3060345/build.log
> > > >
> > > > In brief the build fails at:
> > > >
> > > > + /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/bin/spatch -sp_file demos/simple.cocci demos/simple.c
> > > > init_defs_builtins: /builddir/build/BUILDROOT/coccinelle-1.0.4-2.fc24.ppc64le/usr/lib64/coccinelle/standard.h
> > > > Fatal error: exception Failure("dump: impossible tag (1002)")
> > > >
> > > > When thinking about this bug, don't exclude the possibility that there
> > > > could be a problem in the OCaml compiler.  In Fedora we currently use
> > > > a non-upstream ppc64le code generator, although we are planning to
> > > > replace it soon(-ish) with the new upstream POWER code generator that
> > > > Xavier wrote last year.
> > >
> > > It is actually your code that is crashing :)
> > >
> > > (* Dump an OCaml value into a printable string.
> > >  * By Richard W.M. Jones (rich at annexia.org).
> > >  * dumper.ml 1.2 2005/02/06 12:38:21 rich Exp
> > >  *)
> > >
> > > Maybe you could send the output of man Obj in your version of Ocaml?
> >
> > Not sure I understand - we don't change the Obj man page at all.
> >
> > The implementation of OCaml objects (as in the object header and so
> > on) should also be the same, AFAIK.  Certainly the ppc64le backend
> > doesn't make any changes inside the OCaml compiler core, eg. to the GC
> > or libraries.  It's a 64 bit LE platform, so AFAIK the memory
> > structures ought to be identical to x86-64.
> 
> Actually, I'm not sure to understand which version of OCaml you are using?

4.02.3

The sources (including the non-upstream backend) is:

  https://git.fedorahosted.org/cgit/fedora-ocaml.git/log/?h=fedora-24-4.02

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Julia Lawall]

Here is a patch that should avoid the problem.

julia

diff --git a/parsing_cocci/iso_compile.ml b/parsing_cocci/iso_compile.ml
index 2316e2f..e191ebf 100644
--- a/parsing_cocci/iso_compile.ml
+++ b/parsing_cocci/iso_compile.ml
@@ -14,18 +14,44 @@ isomorphism.  This is allowed if all elements of an isomorphism have only
 one token or if we can somehow match up equal tokens of all of the
 isomorphic variants. *)

+type mcodeinfo =
+    MetaName of Ast.meta_name
+  | Str of string
+  | Cst of Ast.constant
+  | SAOp of Ast0.simpleAssignOp
+  | FixOp of Ast.fixOp
+  | UnOp of Ast.unaryOp
+  | ArOp of Ast.arithOp
+  | LogOp of Ast.logicalOp
+  | CV of Ast.const_vol
+  | Sign of Ast.sign
+  | Struct of Ast.structUnion
+  | Store of Ast.storage
+  | Inc of Ast.inc_file
+
 let sequence_tokens =
-  let mcode x =
-    (* sort of unpleasant to convert the token representation to a string
-       but we can't make a list of mcodes otherwise because the types are all
-       different *)
-    [(Dumper.dump (Ast0.unwrap_mcode x),Ast0.get_pos_ref x)] in
+  let mcode f x =
+    (* have to convert the mcodes to a common type so that we can make a list
+       out of them *)
+    [(f x,Ast0.get_pos_ref x)] in
   let donothing r k e = k e in
   let bind x y = x @ y in
   let option_default = [] in
   V0.flat_combiner bind option_default
-    mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode
-    mcode mcode mcode
+    (mcode (function x -> MetaName (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Str (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Cst (Ast0.unwrap_mcode x)))
+    (mcode (function x -> SAOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> ArOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> FixOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> UnOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> ArOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> LogOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> CV (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Sign (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Struct (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Store (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Inc (Ast0.unwrap_mcode x)))
     donothing donothing donothing donothing donothing donothing donothing
     donothing donothing donothing donothing donothing donothing
     donothing donothing donothing donothing donothing donothing donothing

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

On Wed, Jan 13, 2016 at 05:02:21PM +0100, Julia Lawall wrote:
> Here is a patch that should avoid the problem.

Yes this fixes it for me.

I'm attaching the actual patch as applied/rebased by me since the
patch you sent didn't apply cleanly on top of the coccinelle github
repo.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
-------------- next part --------------
>From 511a9641f1d124fdaf8de7852f61549693065cbf Mon Sep 17 00:00:00 2001
From: Julia Lawall <julia.lawall@lip6.fr>
Date: Wed, 13 Jan 2016 17:02:21 +0100
Subject: [PATCH] Build failure on ppc64le: Failure("dump: impossible tag
 (1002)")

Here is a patch that should avoid the problem.

julia
---
 parsing_cocci/iso_compile.ml | 39 +++++++++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 6 deletions(-)

diff --git a/parsing_cocci/iso_compile.ml b/parsing_cocci/iso_compile.ml
index 53845d1..6fb9fdd 100644
--- a/parsing_cocci/iso_compile.ml
+++ b/parsing_cocci/iso_compile.ml
@@ -14,17 +14,44 @@ isomorphism.  This is allowed if all elements of an isomorphism have only
 one token or if we can somehow match up equal tokens of all of the
 isomorphic variants. *)
 
+type mcodeinfo =
+    MetaName of Ast.meta_name
+  | Str of string
+  | Cst of Ast.constant
+  | SAOp of Ast0.simpleAssignOp
+  | FixOp of Ast.fixOp
+  | UnOp of Ast.unaryOp
+  | ArOp of Ast.arithOp
+  | LogOp of Ast.logicalOp
+  | CV of Ast.const_vol
+  | Sign of Ast.sign
+  | Struct of Ast.structUnion
+  | Store of Ast.storage
+  | Inc of Ast.inc_file
+
 let sequence_tokens =
-  let mcode x =
-    (* sort of unpleasant to convert the token representation to a string
-       but we can't make a list of mcodes otherwise because the types are all
-       different *)
-    [(Dumper.dump (Ast0.unwrap_mcode x),Ast0.get_pos_ref x)] in
+  let mcode f x =
+    (* have to convert the mcodes to a common type so that we can make a list
+       out of them *)
+    [(f x,Ast0.get_pos_ref x)] in
   let donothing r k e = k e in
   let bind x y = x @ y in
   let option_default = [] in
   V0.flat_combiner bind option_default
-    mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode mcode
+    (mcode (function x -> MetaName (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Str (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Cst (Ast0.unwrap_mcode x)))
+    (mcode (function x -> SAOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> ArOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> FixOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> UnOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> ArOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> LogOp (Ast0.unwrap_mcode x)))
+    (mcode (function x -> CV (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Sign (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Struct (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Store (Ast0.unwrap_mcode x)))
+    (mcode (function x -> Inc (Ast0.unwrap_mcode x)))
     donothing donothing donothing donothing donothing donothing donothing
     donothing donothing donothing donothing donothing
     donothing donothing donothing donothing donothing donothing donothing
-- 
2.5.0

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

But note I'm not saying this is not a compiler bug.  We've
had a few.

Is there a test case you could send me?  I will run it on
our ppc64le machine.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[SF Markus Elfring]

> But note I'm not saying this is not a compiler bug.
> We've had a few.

Interesting ?

How do you think about to share any more information
about the last working system configuration
for your special build environment?


> Is there a test case you could send me?

In which kind of test scripts are you interested in?


> I will run it on our ppc64le machine.

Thanks for your offer.

Regards,
Markus

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Richard W.M. Jones]

On Wed, Jan 13, 2016 at 03:54:36PM +0100, SF Markus Elfring wrote:
> > But note I'm not saying this is not a compiler bug.
> > We've had a few.
> 
> Interesting ?
> 
> How do you think about to share any more information
> about the last working system configuration
> for your special build environment?

I'm not sure that it has ever worked on ppc64le.  Looking at our POWER
build systems status:

http://ppc.koji.fedoraproject.org/koji/packageinfo?packageID=919

it looks like it has not built successfully for 3 years, and the last
time it was building that was only on ppc32.

> > Is there a test case you could send me?
> 
> In which kind of test scripts are you interested in?

Anything that would narrow down or potentially reproduce this
particular error, I guess.  Basically if you need me to run any
diagnostic tests, I can.

We also (in partnership with IBM) have a public rack of POWER systems
that anyone can provision (but you have to agree to some legal stuff):

http://research.redhat.com/powerlinux-openpower-development-hosting/

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

[Cocci] Build failure on ppc64le: Failure("dump: impossible tag (1002)")

[Julia Lawall]

I think I can get rid of the reliance on dump for normal execution.  It
won't solve the problem, but it should avoid it.

julia