From : Jeremy Yallop <yallop AT gmail.com>

: Jeremy Yallop <yallop AT gmail.com> To : Malcolm Matalka <mmatalka AT gmail.com>

: Malcolm Matalka <mmatalka AT gmail.com> Cc : David Sheets <sheets AT alum.mit.edu>, Jeremie Dimino <jdimino AT janestreet.com>, Christoph Höger <christoph.hoeger AT tu-berlin.de>, caml users <caml-list AT inria.fr>

: David Sheets <sheets AT alum.mit.edu>, Jeremie Dimino <jdimino AT janestreet.com>, Christoph Höger <christoph.hoeger AT tu-berlin.de>, caml users <caml-list AT inria.fr> Subject : Re: [Caml-list] Save callbacks from OCaml to C

: Re: [Caml-list] Save callbacks from OCaml to C Date : Wed, 3 Feb 2016 16:14:55 -0800

: Wed, 3 Feb 2016 16:14:55 -0800 Authentication-results : mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=yallop AT gmail.com; spf=Pass smtp.mailfrom=yallop AT gmail.com; spf=None smtp.helo=postmaster AT mail-qg0-f54.google.com

: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=yallop AT gmail.com; spf=Pass smtp.mailfrom=yallop AT gmail.com; spf=None smtp.helo=postmaster AT mail-qg0-f54.google.com Ironport-phdr: 9a23:WQ14lBEZZjS1I707cPsmtJ1GYnF86YWxBRYc798ds5kLTJ75oMiwAkXT6L1XgUPTWs2DsrQf27WQ6vGrAzRIyK3CmU5BWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYsExnyfTB4Ov7yUtaLyZ/niKbrqtaJO01hv3mUX/BbFF2OtwLft80b08NJC50a7V/3mEZOYPlc3mhyJFiezF7W78a0+4N/oWwL46pyv+YJa6jxfrw5QLpEF3xmdjltvIy4/SXEGC6G4nAbVmBetxNUCgzG5VmuW5L4riL+teNV1yyTPMmwRrcxD2eM9aBuHT3lkioCJnYI+WXTjdQ42LNSpBamvzRwxofVZMeeM/8oLfCVRs8TWWcUBpUZbCdGGI7pKtJXV+c=

On 3 February 2016 at 12:15, Malcolm Matalka wrote:That's how roots behave, yes: while a value is registered as a root,the value won't be collected. There are (roughly speaking) two typesof root in OCaml: local roots, which persist for the duration of afunction call, and global roots, which persist until explicitlyreleased. A C function binding written by hand must ensure that OCamlvalues passed to it as arguments are registered as local roots, sothat if a collection occurs while the function is running the valueswon't be prematurely collected.A C binding written using ctypes can generally ignore the matter ofroots. That's partly because ctypes takes care of root registration,but also because most types passed between OCaml and C in a ctypesbinding are C values, not OCaml values. For example, if you want topass a structure with several fields between OCaml and C there are twoapproaches. One approach is to represent the structure as an OCamlrecord, which involves accessing the fields of the value in your Cbinding using various macros, taking care to register values as rootsto protect them from the GC. The other approach is to represent thestructure as a C struct, which involves accessing its fields in OCamlusing the functions ctypes provides. (If you enjoy programming in anuntyped dialect of C with ubiquitous concurrency, you'll probablyfavour the first approach. If you prefer programming in OCaml thenthe second approach might have some appeal.)Using the C value representation for values that cross the C-OCamlboundary generally works well, but when things become higher-order,the situation changes a bit. When a C library expects to be given afirst-order value such as a struct we have to give it a struct withthe appropriate layout, since C functions can directly access therepresentation of values. However, when the library expects afunction pointer we have a bit more freedom, since the representationof functions isn't accessible -- in fact, the only thing that can bedone with a function pointer, besides passing it from place to place,is calling it. This freedom means that we can pass an OCaml function,suitably packaged up, where a C function pointer is expected.Passing OCaml functions to C as function pointers raises someinteresting issues relating to object lifetime and the garbagecollector. The main difficulty arises from the fact that once youpass a function pointer to a C library there's no way of knowing howlong the library holds on to it: for example, the library mightdiscard the function pointer when the call into the library returns,or it might store the function pointer in a global variable to beretrieved and called later. In order to prevent the associatedfunction from being collected prematurely, some kind of action isneeded on the OCaml side, whether registering a global root, orensuring that the function is reachable from the OCaml program.Storing some kind of references to the functions in a place that thecollector can see is essential to prevent the functions from beingcollected prematurely. The situation is the same whether you usectypes or write bindings by hand.Storing the functions in a table, and removing them automaticallyafter they're called is one approach. An alternative is to use thenew Ctypes.Roots module, which will be available in the next release:In that case it sounds like there'll be an overhead of up to a few megabytes.