Emacs Lisp's future (was: Guile emacs thread (again))

From: Stefan Monnier Subject: Emacs Lisp's future (was: Guile emacs thread (again)) Date: Tue, 16 Sep 2014 11:50:54 -0400 User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux)

> So this email is partly a: > - What now? What's the chance of work towards guilemacs moving over to > an official emacs git branch, and that port happening, anytime soon? > - Is anyone running it? How's it going for you? Good questions. I've had the opportunity to think a bit more about Emacs Lisp and its possible evolution and I'm still not sure what to think about it. I see a few different options for Emacs Lisp. First, of course we can keep on evolving Elisp on its own. This has worked OK for the last 30 years, so it's not such a terrible choice. The main problems I see with that: - Elisp is slow and as CPUs aren't getting faster, its slowness makes itself noticed more often. - Lack of some features, most notably FFI and concurrency. - Lack of manpower. This last point is for me the strongest motivation to try and move to some other system, where we could use other people's work. One such option is Guile-Emacs. This presumably would give us a faster implementation (at least in theory, their bytecode is significantly more efficient), would give us an FFI, and would give us more manpower since we'd be benefiting from the work done on Guile. Note that while Guile does come with support for threading, it doesn't immediately let us use concurrency in Guile-Emacs, because of all the issues of synchronizing access to shared data, with all the existing Emacs code (both C and Elisp) assuming that this problem doesn't exist. IOW, language support for concurrency is just a first step on the way to letting Emacs Lisp use concurrency. Another detail that needs to be spelled out is the difference between the language and its implementation. Guile-Emacs provides 2 languages: Emacs Lips and Scheme (well, it also provides a few more, but that's not important). Many people are thinking "cool, so I'll be able to write extensions in Scheme", but I'm not sure defining Emacs as "this editor that comes with N extensions languages" is a good idea. One of the main reasons for Emacs's enduring success is its large set of third party packages so obviously we can't drop support for Elisp any time soon. And as much as I like Scheme, I'm very much unconvinced that it's really so much better that it's worth converting packages from Elisp to Scheme. So if we go for Guile-Emacs, we'll be stuck with Guile, i.e. we'd have (old and new) packages that use Elisp, new packages that use Scheme, maybe yet other new packages that use, say, Javascript (or some other language support by Guile). That would make the work of Emacs (and GNU ELPA) maintenance harder. And of course, if Guile's own manpower dries up, Emacs would be forced to keep supporting Guile, which is more work than supporting just Elisp. So, I think that ideally, we'd want to stick to Elisp, or some evolution thereof. Sadly, I don't see how to evolve Elisp into Scheme: they are closely related languages, but the differences are large enough that it seems hard to reconcile them. The only standard language into which Elisp can evolve, AFAICT, is Common Lisp. [ Now some readers get disappointed, while some others become excited. ] There are some incompatibilities between the two languages, but I can imagine working them out over the years, or even living with them without too much trouble, such that we could use Common-Lisp libraries in Emacs. Of course, that's for the language side, but on the implementation side, I don't really know what Common-Lisp implementation we could re-use (both GNU implementations are dormant, so there's no manpower for us tap into). Still: there are many Common-Lisp implementations out there, so there's probably one that could work for us. Stefan