Re: [cocci] Very long compile times on parsing_cocci/parser_cocci_menhir.ml

["Richard W.M. Jones" <rjones@redhat.com>]

On Thu, May 02, 2024 at 07:02:30PM +0200, Julia Lawall wrote:
> 
> 
> On Thu, 2 May 2024, Richard W.M. Jones wrote:
> 
> > On Thu, May 02, 2024 at 06:39:34PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Thu, 2 May 2024, Richard W.M. Jones wrote:
> > >
> > > > On Thu, May 02, 2024 at 11:44:00AM +0200, Julia Lawall wrote:
> > > > >
> > > > >
> > > > > On Thu, 2 May 2024, Richard W.M. Jones wrote:
> > > > >
> > > > > > OCAMLOPT  parsing_cocci/parser_cocci_menhir.ml
> > > > > >
> > > > > > This single file takes a very long time to compile, about 34 minutes
> > > > > > on my 2 year old Intel laptop.  It's quite large too ...
> > > > > >
> > > > > > 139201 ./coccinelle-1.2/parsing_cocci/parser_cocci_menhir.ml
> > > > > >
> > > > > > I've observed in the past that the OCaml compiler doesn't handle
> > > > > > single large files well.  It seems to have some O(n^k) behaviour.
> > > > > >
> > > > > > Is this a known issue?
> > > > >
> > > > > I agree that it's slow, but I haven't seen it take 34 minutes.
> > > >
> > > > I just tried it on a much faster AMD desktop machine, but essentially
> > > > the same OCaml and menhir, and it still takes a very long time to
> > > > compile this one file: 17:35 minutes
> > > >
> > > > I tried it on a (different) machine with OCaml 4.11.1 and that was
> > > > considerably faster, taking under a minute.
> > >
> > > I tried it on an Intel server and it takes a few seconds.  OCaml version
> > > 5.1.1.  menhir, version 20200123
> >
> > Interesting, thanks Julia.  I'll poke this further to see
> > if I can find out what the causative factor is.
> 
> wc -l parser_cocci_menhir.ml gives 139201 parser_cocci_menhir.ml

It turns out this happens because Fedora enables "flambda" in the
OCaml compiler (ie. ./configure --enable-flambda).

flambda is "a series of optimisation passes provided by the native
code compilers as of OCaml 4.03" which "make it easier to write
idiomatic OCaml code without incurring performance penalties".  It's
not enabled by default, which probably explains why this problem is
not seen on Debian.

More here:
https://ocaml.org/manual/5.1/flambda.html

Anyway it's not a cocci problem!

I will discuss with Fedora if we want to keep this option enabled or
drop it.  It's not a show-stopper that Coccinelle takes a long time to
compile, just a bit annoying.  It doesn't seem to affect Coccinelle in
any other way (eg. making it faster/slower at runtime) as far as I can
tell.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit