A simple SOAP interface in Lisp

SOAP is a powerful but very complicated protocol for doing remote procedure calls. Its complexity makes it a daunting task even to do simple things.

As part of a project at Franz Inc. we needed code to make Java and Lisp communicate using the SOAP protocol. There would be a number of services written in Java and and a number of services written in Lisp and calls had to work with Lisp calling Java, Java calling Lisp, Java calling Java and Lisp calling Lisp.

Java has a web service library which we couldn't modify so we had to make Lisp's web services obey the conventions used by Java to expose a web service. Specifically, the Java code was using the JAXWS-2.0 library for web services.

We wrote the Lisp code for calling the first Java web service and it was a tedious task. Not relishing the thought of writing many more interfaces we did what Lisp programmers do, we wrote a program to write our interface code.

This code, which is provided in a modified form here as part of this note, was specific to our project. It is not part of Allegro CL. Users are free to take it and adapt it for their own use. Note it is provided As Is, with no waranties expressed or implied, etc. etc. Note that your Lisp must have all recent SOAP updates and patches in order for the code to work. See sys:update-allegro (the function for downloading patches) for information on getting updates.

We call our our program to write Lisp SOAP code ssoap, meaning Simple SOAP. Below we give an example showing how it works.

Because we wrote the code for our specific project, it only supports passing the data types needed for that project. Users interested in the code must modify and extend it as needed to suit their needs.

ssoap achieves simplicity by eliminating many (perhaps most) of the choices you have when writing a SOAP application. Our goal was not to build a new complete soap interface, just to make simple one which could do the necessary things and to talk to Java applications.

Java's web services do the following to expose themselves.

If, say, the web service is located at http://machine.com:8080/MyTask/MyTaskService then an http GET request to that URL will return a 200 status code response. What it returns with that response is irrelevant for our purposes (what it in fact does is send back a description of the methods that service accepts). All we need is the information that if the web service is up and running, a 200 response is returned for a normal http GET command. If the web service is located at http://machine.com:8080/MyTask/MyTaskService then the http request for http://machine.com:8080/MyTask/MyTaskService?wsdl will return an xml file which is the wsdl for this service. This is critical because often Java clients will read the wsdl at parse time to know how to call the service.

The ssoap code arranges for both of the above behaviors to occur for the soap services it starts.

In a SOAP call there are two actors: the client and the server. The server sits idle and listens for a call. The client makes the call and optionally passes data to the server. The server performs the requested action and then notifies the client that it is finished, optionally passing back data to the client.

In ssoap we define a service. A service consists of one or more methods. Each method has a client side and a server side.

We will generally know when we define the service whether this is a service we will be implementing in Lisp or whether we just want access to this service as a client from Lisp.

With ssoap you can even do both: build a service in Lisp and also provide the client interface to the service. In fact if you want to implement a service in Lisp you'll likely want to define the client interface in Lisp as well so that you can more easily test your Lisp service.

ssoap is used in this way:

you define your service in a file. you start lisp and load ssoap. you load your service definition and this causes the interface definition file to be written. This file contains the definition of your service written for the ACL SOAP module. you compile and load in that interface file If you're just using the Service as a client then you are done. If you're implementing the Service as a server you now load in your code that does the work of the service.

Let's look at an example, this one for a service that does conversion between Celsius and Fahrenheit. We put the following forms in a file called converter-ssoap.cl (the contents are in the dowloadable ssoap.cl file in a commented section):

(in-package :user) (let ((sss (make-ssoap-service :name "Converter" :package :user :target-namespace "http://webservice.converter.franz.com/" :host "127.1" :port "8088" ;; optional, specify if you want to create a server :server 'start-converter-ss-soap-server :prefix 'converter ;; used to construct names :messages '(("CelsiusToFahrenheit" :in (("celsius" :float)) :out (("fahrenheit" :float)) :client (converter-ss-celsius-to-fahrenheit &key url celsius) :server (converter-ss-server-celsius-to-fahrenheit &key celsius) ) ("FahrenheitToCelsius" :in (("fahrenheit" :float)) :out (("celsius" :float)) :client (converter-ss-fahrenheit-to-celsius &key url celsius) :server (converter-ss-server-fahrenheit-to-celsius &key fahrenheit) )) ))) (generate-interface-code-file sss "converter-interf.cl") (generate-wsdl sss "converter.wsdl") )

The arguments to make-ssoap-service are as follows.

name : the :name should be a capitalized word (by convention). Here we chose "Converter" so the URL path to our service will be /Converter/ConverterService (by the convention used by the Java web service code).

: the :name should be a capitalized word (by convention). Here we chose "Converter" so the URL path to our service will be /Converter/ConverterService (by the convention used by the Java web service code). package : the :package argument should be the same package as is mentioned at the top of the file (in this case :user). When we generate the interface file (below) this is the package that will be mentioned in the in-package form at the beginning of that file.

: the :package argument should be the same package as is mentioned at the top of the file (in this case :user). When we generate the interface file (below) this is the package that will be mentioned in the form at the beginning of that file. target-namespace : the :target-namespace is the XML namespace in which we'll define our service. It should be of the form shown in the example. Should we generate java code use or serve this service, the interface code will be put in the com.franz.converter.webservice java package, so keep that eventual java namespace to java-package conversion in mind when chosing a target-namespace. host and port : the :host and :port arguments are our best guess as to where the service will be hosted. This information only used if you create a wsdl file from this definition using generate-wsdl as is shown in the example. The wsdl file, if generated, is not used by the Lisp code. It's only present to allow other soap systems (e.g. Java) to be clients or servers of this service.

: the :target-namespace is the XML namespace in which we'll define our service. It should be of the form shown in the example. Should we generate java code use or serve this service, the interface code will be put in the com.franz.converter.webservice java package, so keep that eventual java namespace to java-package conversion in mind when chosing a target-namespace. server : if you want Lisp to serve this service then you specify a value for the :server argument. ssoap will then create a function to start the server and will give it the name you specify. The function will take one optional argument, a port number, which is the port on which to start the service. If you don't specify a port then the operating system will chose a free port. The return value from starting the service is a string holding the url of the service (including of course the port where the service is running).

: if you want Lisp to serve this service then you specify a value for the :server argument. will then create a function to start the server and will give it the name you specify. The function will take one optional argument, a port number, which is the port on which to start the service. If you don't specify a port then the operating system will chose a free port. The return value from starting the service is a string holding the url of the service (including of course the port where the service is running). prefix : the :prefix argument specifies a name that the ssoap will use when generating symbol names in the interface file. Chose wisely and you'll find debugging easier when looking at stack backtraces.

: the :prefix argument specifies a name that the will use when generating symbol names in the interface file. Chose wisely and you'll find debugging easier when looking at stack backtraces. messages: the :messages argument is where all the soap methods are defined.

Each method begins with the name of the method and then is followed by a sequence of arguments.

The :in and :out arguments specify the values sent to and received from the web method. Each value is given a name (that ends up being an XML element name so choose simple names), and a type. So far ssoap only support four types: :int, :string, :float, :base64Binary. Other types can be added to the ssoap code as necessary (by modifying the functions lookup-xsd-type and lookup-xsd-type-symbol).

The :client argument is optional and if present tells ssoap to create a client function in Lisp to call this method. The value of the :client argument is the name and signature of the method to call. The signature will always be &key followed by url and then followed by symbols with the same names as the :in arguments to the function. Since the signature can be computed automatically why are we forcing the user of ssoap to specify it? The reason is that it then makes it clear to the person viewing this service definition file how the client function should be called. The required url argument to the client function is the location of the service. The lisp client of the service must know where the service is.

The :server is optional and need only be specified if you want Lisp to serve this service (in which case you've also specified the :service argument as described above). The server argument specifies the signature of a function that you must write to perform the action of this method. Again the signature can be computed from the :in parameters of the method by turning those input arguments into keyword arguments. The ssoap module forces you to specify it though so that the defintion file explicitly reminds you of the signature of this method.

Once the make-ssoap-service function is run it builds the ssoap-service object and we then pass it to generate-interface-code-file, specifying a file to be written with this interface code. Next well want to compile and load in this file. Finally we write out a wsld file which we may need if we're going to interface with Java, for example, but more on this later.

Before we work with Java let's show how we can use ssoap to make Lisp talk with Lisp.

Our example above includes both client and server functions so we'll use it to create both a server and client interface to that server.

We need one more piece of code to run the example. We need to implement the server side for the service we've defined.

Let's do just one, the server for CelsiusToFahrenheit.

("CelsiusToFahrenheit" :in (("celsius" :float)) :out (("fahrenheit" :float)) :client (converter-ss-celsius-to-fahrenheit &key url celsius) :server (converter-ss-server-celsius-to-fahrenheit &key celsius) )

We've aleady specified the signature of the function:

:server (converter-ss-server-celsius-to-fahrenheit &key celsius)

Now we define it using that signature

(defun converter-ss-server-celsius-to-fahrenheit (&key celsius) (list "fahrenheit" (+ 32 (* 9/5 celsius))))

The value returned from a function implementing a soap service is in property list format, a list of alternating names and values.

In this case we see from the definition of the function

:out (("fahrenheit" :float))

we return just one value and its name is "fahrenheit".

Now we're ready to run the example.

compile and load ssoap.cl, the module we're describing here. load converter-ssoap.cl, the example file above. There is no need to compile this file. At this point, two files have been generated: converter-interf.cl and converter.wsdl. If you're interested you can view the wsdl file but we don't need it now. compile and load the converter-interf.cl file compile and load in the file holding the implementation of the service (in our case the definition of converter-ss-server-celsius-to-fahrenheit above)

No we can start the SOAP server for the service

cl-user(4): (start-converter-ss-soap-server 8088) #<net.xmp.soap:soap-aserve-server-string-in-out-connector @ #x1001e11ab2> 8088 "http://10.100.40.243:8088/Converter/ConverterService"

The third return value is the URL for the service. Note you will get a different value.

We can now call the service as a client by using the :client method we declared (we use the URL value returned above -- if you run this example, you should use the value you get which will be different):

cl-user(7): (converter-ss-celsius-to-fahrenheit :url "http://10.100.40.243:8088/Converter/ConverterService" :celsius 100) (converter-ssoap-gen-package::CelsiusToFahrenheitResponse (:fahrenheit 212.0)) nil cl-user(8):

The return value of the client call is XML expression returned from the server, parsed into what we call lxml format. It's easy to find in it the the answer requested from the service.

Finally we'll consider how to communicate with Java. Suppose you have a Java SOAP server and you wish to wish to call it from Lisp using ssoap. At present the only way to do this is for you to write a ssoap definition by hand based on the wsdl for the service. We hope to automate this at some time in the future.

In order to use Java as a client to an ssoap server you need to create a web client interface in Java. There are various tools in Java to do this. The Netbeans IDE makes it particuarly easy. You can ask Netbeans to create client interface objects from either a wsdl file or from an active url (such as http://10.100.40.243:8088/Converter/ConverterService?wsdl).

The client interface is hardwired to call the service at the location specified in the wsdl file (although with a bit of work you can access services whose location you learn about at run time).

We used Netbeans to create a client interface based on the wsdl created by our ssoap definition of the Converter service. Then we asked Netbeans to insert a call to the CelsiusToFahrenheit method of that service. We specified an initial value to test and added a print statement to show the results and we then had this main program:

package converter; /** * * @author jkf */ public class Main { /** * @param args the command line arguments */ public static void main(String[] args) { // TODO code application logic here try { // Call Web Service Operation com.franz.converter.webservice.ConverterService service = new com.franz.converter.webservice.ConverterService(); com.franz.converter.webservice.Converter port = service.getConverterPort(); // TODO initialize WS operation arguments here float celsius = (float) 100.0; // TODO process result here float result = port.celsiusToFahrenheit(celsius); System.out.println(celsius + " Celsius is " +result + " Fahrenheit"); } catch (Exception ex) { // TODO handle custom exceptions here System.out.println("calling service got exception " + ex); } } }

when this is run it prints

100.0 Celsius is 212.0 Fahrenheit

In summary the ssoap module proves a simple way to describe a soap service and to write the necessary glue code to start that service and to call that service as a client. Furthermore it implements server behavior expected by java web services so that it will interoperate with them. Again, the code is availabe here, and will only work if all SOAP updates and patches have been loaded.