Re: Current LibXL Status

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2016-02-18 at 17:39 +0000, Ian Campbell wrote:
> On Thu, 2016-02-18 at 17:26 +0000, George Dunlap wrote:
> 
> > According to them, the ocaml garbage collector never frees memory.  It
> > grows its own internal heap as necessary, but it never reduces the
> > size of its heap.
> 
> Assume that's true: Wow!

https://github.com/ocaml/ocaml/blob/trunk/byterun/compact.c#L374
is a call by do_compaction to caml_shrink_heap at
https://github.com/ocaml/ocaml/blob/trunk/byterun/memory.c#L408
which calls caml_free_for_heap at
https://github.com/ocaml/ocaml/blob/trunk/byterun/memory.c#L290
which certainly looks like it is returning memory to the malloc pool.

Maybe this never happens in practice for some reason, or perhaps the
"byterun" in those paths is relevant? I couldn't find another heap
implementation which would be associated with native binaries.

http://caml.inria.fr/mantis/view.php?id=5389 also seems to suggest that it
is expected for the heap to shrink (i.e. it was considered a bug that it
did not in this case).

Ian.

> 
> Although, I presume this is a factor of the current/particular
> implementation, rather than a fundamental property of an ocaml runtime,
> so
> in theory it could change (and we have here a good real world argument
> why
> it perhaps should do!)
> 
> > So no -ENOMEM call or OOM callback can make a failed malloc succeed.
> > 
> > One might then ask if libxl could simply allocate memory *from the
> > ocaml heap* itself.  It turns out that is also not tenable: Data in
> > the ocaml heap is stored in a heavily coded format.  (For example
> > integers are stored as (n*2+1), so that pointers can all be even and
> > non-pointers can all be odd.)
> > 
> > The only thing they said might be improved is:
> > 
> > 1.  To know that libxl would never call exit() for any other reason
> > (which it seems is true)
> > 
> > 2. To  have a callback in OOM conditions.  It's unlikely the process
> > as a whole could do anything but exit, but the ocaml runtime itself
> > might still have internal heap available, which would allow it to exit
> > more gracefully.
> > 
> > I think if Haskell or any library *is* capable of integrating with the
> > garbage collector, we'll have to wait until someone who understands
> > the language actually writes bindings and can ask for something they
> > know to be useful for that language.
> 
> Agreed.
> 
> Hopefully it is possible to make the callback without needing to malloc
> anything!
> 
> If I were adding an API for #2 to libxl (which must therefore be stable)
> I
> think I might be inclined to allow for the possibility of the callback
> returning "please retry" since it would be inconvenient in the usual API
> stability ways to retrofit it, I would guess, but really until a language
> even exposes that possibility we don't really know if it is worthwhile
> doing so.
> 
> Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel