let

Program generators, as programs in general, greatly benefit from compositionality: the meaning of a complex expression is determined by its structure and the meanings of its constituents. Building code fragments using MetaOCaml's brackets and escapes is compositional. For example, in

let sqr: int code -> int code = fun e -> .<.~e * .~e>.

sqr e

e

e

let make_incr_fun : (int code -> int code) -> (int -> int) code = fun body -> .<fun x -> x + .~(body .<x>.)>. let test1 = make_incr_fun (fun x -> sqr .<2+3>.) (* .<fun x_24 -> x_24 + ((2 + 3) * (2 + 3))>. *)

make_incr_fun body

Compositionality lets us think about programs in a modular way, helping make sure the program result is the one we had in mind. We ought to strive for compositional programs and libraries. The trick however is selecting the `best' meaning for an expression. Earlier we took as the meaning of a code generator the exact form of the expression it produces. Under this meaning, compositionality requires the absence of side-effects. Our test1 is pure and so its result is easy to predict: it has to be the code of a function whose body contains two copies of the expression 2+3 . Although the confidence in the result is commendable, the result itself is not. Duplicate expressions make the code large and inefficient: imagine something complex in 2+3 's place. Furthermore, 2+3 in test1 does not depend on x and can be lifted out of the function's body -- so that it can be computed once rather than on each application of the function. In short, we would like the result of test1 to look like

let test_desired = .<let t = 2+3 in fun x_24 -> x_24 + (t * t)>.

test1

test_desired

test1

2+3

2+3

read_config "foo"

read_config

IO

read_config

We change the meaning of code generators: sqr e now means the multiplication of two copies of e or of the result of evaluating e . The meaning is the generated code with possible let-expressions. Compositionality now permits effectful generators, whose side-effect is let-insertion. MetaOCaml could offer simplest such generators as built-ins. Adding more built-ins to MetaOCaml -- just like adding more system calls to an OS -- makes the system harder to maintain and ensure correctness. Generally, what can be efficiently implemented at the `user-level' ought to be done so. Let-insertion can be written as an ordinary library, relying on the delimited control library delimcc .

The first attempt at let-insertion takes only a few lines:

open Delimcc let genlet : 'w code prompt -> 'a code -> 'a code = fun p cde -> shift p (fun k -> .<let t = .~cde in .~(k .<t>.)>.) let with_prompt : ('w prompt -> 'w) -> 'w = fun thunk -> let p = new_prompt () in push_prompt p (fun () -> thunk p)

genlet

with_prompt

with_prompt (fun p -> sqr (genlet p .<2+3>.)) --> (* evaluating genlet *) with_prompt (fun p -> sqr ( shift p (fun k -> .<let t = .~(.<2+3>.) in .~(k .<t>.)>.))) --> (* evaluating shift, capturing the continuation up to the prompt, binding it to k *) with_prompt (fun p -> let k = fun hole -> push_prompt p (fun () -> sqr hole) in .<let t = .~(.<2+3>.) in .~(k .<t>.)>.) --> with_prompt (fun p -> let k = fun hole -> push_prompt p (fun () -> sqr hole) in .<let t = 2+3 in .~(k .<t>.)>.) --> (* applying the captured continuation k *) with_prompt (fun p -> .<let t = 2+3 in .~(push_prompt p (fun () -> sqr .<t>.))>.) -->* (* evaluating (sqr .<t>.) *) with_prompt (fun p -> .<let t = 2+3 in .~(.<t * t>.)>.) -->* .<let t = 2+3 in t * t>.

genlet

.<2+3>.

with_prompt

2+3

genlet

with_prompt (fun p -> make_incr_fun (fun x -> sqr (genlet p .<2+3>.))) (* .<let t_17 = 2 + 3 in fun x_16 -> x_16 + (t_17 * t_17)>. *)

2+3

with_prompt (fun p -> make_incr_fun (fun x -> sqr (genlet p .<.~x+3>.))) (* BEFORE N100: .<let t_17 = x_16 + 3 in fun x_16 -> x_16 + (t_17 * t_17)>. *) (* BER N101: exception pointing out that in .<.~x+3>. the variable x escapes its scope *)

x

We have just seen the code generation with control effects, that let-insertion is highly desirable and highly dangerous, and that in the present MetaOCaml, it is finally safe. It is safe in the following sense: if the generator successfully finished producing the code, the result is well-typed and well-scoped.

Although the naive attempt to program let-insertion works and is now safe, it is not convenient. One has to explicitly mark the let-insertion place. When several places are marked, we or the user have to choose. We want to automate such choices. We would like the result of test1 to be the desired code test_desired , with let-bindings. The operations sqr and make_incr_fun -- which are programmed by the DSL designer -- may be re-defined. However, test1 should be left as it was. It is written by the end user, who should not care or know about let-insertion's taking place, let alone pass prompts around.

Recall that trying to insert let at a wrong place raises an exception. This dynamic exception lets us program genlet so to try the insertion at various places -- farther and farther up the call chain -- until we get an exception. The best place to insert let is the one that is farthest from genlet while causing no exceptions. We arrive at the following simplified interface for the let-insertion:

val genlet : 'a code -> 'a code val let_locus : (unit -> 'w code) -> 'w code

let_locus

let

genlet

sqr

make_incr_fun

sqr

make_incr_fun

let sqr e = sqr (genlet e) let make_incr_fun body = let_locus @@ fun () -> make_incr_fun @@ fun x -> let_locus @@ fun () -> body x

sqr

make_incr_fun

test1

let test1 = make_incr_fun (fun x -> sqr .<2+3>.) (* .<let t_17 = 2 + 3 in fun x_16 -> x_16 + (t_17 * t_17)>. *) let test2 = make_incr_fun (fun x -> sqr .<.~x + 3>.) (* .<fun x_18 -> x_18 + (let t_19 = x_18 + 3 in t_19 * t_19)>. *)

test2

We have demonstrated for the first time the self-adjusting, safe and convenient let-insertion with static guarantees. The scope extrusion check meant to crash bad generators surprisingly helps implement good ones -- more convenient than those possible otherwise.