Re: Dynamic loading progress

From: Aurélien Aptel Subject: Re: Dynamic loading progress Date: Thu, 9 Oct 2014 19:08:47 +0200

Good news everyone! As of my last commit (7a38270cd) dynamic loading is working pretty well. I've written a few toy modules in the "modules" folder. They can be used as examples and cover various use-case. - The `fmod' module simply expose the fmod(3) function from the math lib. - The `elisp' module evaluates code written in Lisp from C. - The `opaque' module show how any C structs/type can be embedded in a Lisp_Object. - The `curl' module can actually be useful as it links against libcurl and lets you fetch any http url supported by curl. You can use the same curl handle for multiple request in order to reuse the connection when possible (faster!). * Usage Dynamic loading can be enabled/disabled via the configure script. libltdl/ltdl.h is all you need to make it work. I've turned the option on by default (--with-ltdl). # build emacs $ git clone https://github.com/aaptel/emacs-dynamic-module.git emacs-dyn $ cd emacs-dyn $ ./autogen.sh $ ./configure $ make # build all the modules $ for i in modules/*; do make -C "$i"; done # run emacs! (no init file) $ ./src/emacs -Q Switch to *scratch* to test interactively the curl module: (add-to-list 'load-path "path/to/emacs/modules/curl") (require 'curl) => t ;; alternatively you can use ;; (load "path/to/curl/curl.so") ;; or ;; (load-module "path/to/curl/curl.so") (defvar c) (setq c (curl-make)) ;; allocate/init a curl handle => #<save-value <pointer 0x1cf56d0> <unused> <unused> <unused>> (curl-fetch-url c "http://example.com") ;; make and store request => t (curl-content c) "<!doctype html> <html> <head> <title>Example Domain</title> ..." (curl-free c) ;; free the curl handle => t I've just tested the build procedure above before posting and *of course* for some reason a clean emacs build fail on something completely unrelated. Even at my first commit where I didn't make any change yet. make[1]: *** No rule to make target 'ctags.c', needed by 'ctags'. Stop. I suppose it comes from my setup? Gahh.. Anyway... * Implementation details and potential issues ** Loading The actual loading is done in Fload_module in lread.c. It is called from Fload when appropriate. See my previous emails. When dynamic modules are enabled, the `load-suffixes' variable has the additional .so suffix. This variable is used by `load' which in turn calls openp() to do the actual search in `load-path'. Adapting `load' to support modules made `require' and `provide' work with no additional work. Nothing is preventing a module from being loaded several times apart from the `features' mechanism. This might be problematic. ** Documentation I have updated several documentation related functions in order to support multiple DOCFILE (each module has its own). See my previous emails. The xref on function documentation (go to definition) is currently broken. I have not looked into it (issue #6). I have not fixed the issue (#1) Stefan mentioned earlier about a memory leak in the GC on the subr doc field because I *still* don't get it :( Any help is welcomed. ** Opaque user types Adding a new proper type is too much work for a module. You need to provide a reader, a printer, add a bunch of enums, etc. I've reused the Save_Value type to store arbitrary pointers. The object hierarchy is as follows: Lisp_Object > Lisp_Misc > Lisp_Save_Value > void* pointer A major problem with this is that you can't make type predicates e.g. `curl-handle-p'. A solution would be to make a new generic user type composed of a user pointer and a cookie or some sort of ID we can test. The space has to be large enough to limit collisions. ** Resource management An allocated curl handle has to be explicitly freed with `curl-free'. Could we extend the GC to provide some sort of destructor mechanism? Just a thought. Also, Lisp_Objects contained in a user type are not protected from the GC. This can be a problem. ** Native type size EMACS_INT_MAX is not always equal to INT_MAX depending on the architecture. Libraries can often use special value to signal something (e.g. (unsigned)-1 which is 0xffff..) which might not be fit in a Lisp type. What can we do about it? Is it really a problem? Emacs Lisp provides other way to signal errors and a module doesn't have to expose low-level stuff like this. ** Packaging Right now each module has its own Makefile and is built separately from emacs. We need to think of a way to package modules for distribution that is practical and takes into account what the module needs from the Emacs build (which configure option should be enabled to build module xxx?). ** Portability The .so suffix is hardcoded but via libltdl loading should work on most systems.

reply via email to

