Update: there were some technical inaccuracies in the article, which I have fixed.

Preface

A couple of weeks ago, I was asked to create in OutSystems an interface with a REST API of an external supplier. Except for a terrible lack of functional documentation on the side of the REST API’s supplier, the process was pretty straightforward until I hit a stumbling block: one of the methods, for uploading documents, used something called “multipart/form-data”, which is not natively supported by the Platform. Being pretty active on the OutSystems Community Forum, I have regularly seen people running into this problem when trying to consume a REST service (see e.g. here, here and here), but had never experienced it myself, until now. So I did two things: first, I created a Community Idea for OutSystems to support it natively, and secondly, I came up with an idea of how to satisfy the REST API with minimal impact for my implementation of its interface, which I’ll describe in this article.

REST in OutSystems

The first thing I want to look at is the way OutSystems implements REST. REST is a protocol on top of HTTP (the protocol used for retrieving web pages), and typically (though not exclusively) uses JSON to communicate data. When sending data via HTTP, the sender must specify the type of content it is sending. Since OutSystems support JSON only¹, it sends “application/json” in the HTTP “Content-Type” header. In the body of the message, the actual JSON is sent, in a compact manner (no line breaks), like this:

{"name":"Test for POST","type":"PACKAGE","language":"en","description":"This is a test for posting a package"}

This works for probably 95% of REST methods, so it’s in line with OutSystems philosophy of supporting the most commonly used software patterns, without the want to support them all.

Multiwhat?

So what is this “multipart/form-data” I talked about earlier, and why is it used? First some background. “Form data” is HTTP terminology for any data a user enters on a web page (“HTML form”) and which is subsequently sent (or “posted”) to a web server via HTTP. Typically a form contains only simple input fields, and those are sent in a format that you might recognize, as it is the same format as used for passing parameters to web screens: it consists of name/value pairs, separated by an assign character (“=”), concatenated by an ampersand (“&”) with special characters replaced by a percentage sign and their hexadecimal character code (e.g. “%20” for space):

name=This%20is%20a%20test&language=en

The HTTP content type for this format is “application/x-www-form-urlencoded”. This works fine for simple input fields, but web pages may also allow the user to upload one or more files. To support this, another format was created in the early days of the web, and it is known by its content type, “multipart/form-data”. So in essence, multipart/form-data is a way for a browser to send one or more files to a web server (it was later co-opted for sending attachments in e-mail, and even later for REST). The exact format for sending multipart/form-data is rather arcane (as are most early web protocols), but fortunately for us it’s mostly text only, so it’s easy to read and manipulate.

First, instead of just the content type, a client (like a browser or REST consuming app) sending multipart/form-data sets the HTTP Content-Type to “multipart/form-data”, but also needs to specify something called a boundary. This is a unique text string that must be guaranteed not to appear anywhere in the message, so it can signal the several sections or parts (it’s multipart, after all). It is customary to start the boundary with a string of dashes followed by some kind of GUID, though a client is free to set the boundary to whatever it likes (within limits), as long as the rule I stated above is met. The boundary is added after the content type with a semicolon, followed by the text “boundary=”, followed by the boundary itself, between double quotes:

Content-Type: multipart/form-data;boundary="MyBoundary"

The body of the message consists of one or more section, each part starting with two dashes ( -- ) followed by the boundary text. On the next line, there’s the text Content-Disposition: form-data; name="name" , where name between quotes is the name of the item. If the section contains file data, the line also contains the filename, like this:

--MyBoundary

Content-Disposition: form-data; name="myFile" filename="myfile.docx"

After this line there’s a blank line, followed by the value associated with “name”. For simple input fields this is the value of the input field, for files this is its binary data. Directly after the data, a new section starts (without a blank line in between). After the last section, to signal the end of the multipart/form-data message, the boundary is added again, but this time not only starting, but also ending, with two dashes:

--MyBoundary--

Of course, I’m not the first one to attempt to solve the problem of sending multipart/form-data. Forge Components for various REST APIs have solved it in a number of different ways, and there even is a Forge Component for sending multipart/form-data independently from REST. But none of what I have seen so far was entirely to my liking. So I decided to create an implementation that suited my needs (and my taste for aesthetics).

Implementation details

Whatever implementation one choses, in the end one needs to do two things: send the right Content-Type header, and send the correctly formatted body (as explained above). If you include files containing binary data (i.e. anything other than a text file), you need to declare the REST API Method having a Parameter with Data Type “Binary Data”, and its “Send In” Property set to “Body”. This allows sending the binary data unmodified, which is what we want. If also prevents the Platform from adding a Content-Type tag in the Header, which you then need to add (via a Parameter with “Send In” set to “Header”). The (small) downside is that in order to compose the message, you need the Binary Concat Forge Component in order to be able to concatenate the various parts, as well as Actions from the BinaryData Extension (which is a System Component, so already installed) to convert the text-based parts of the message to binary before concatenating it with the binary data of the files.

Another approach, which only works when the message does not contain any binary data (e.g. when sending text files only), is to declare the REST API Method as a “normal’ JSON one, and modify the message before it is sent to the REST service. In OutSystems, this is done by defining an OnBeforeRequest Action. An OnBeforeRequest Action is defined in the Advanced Properties of a REST API. Chose “New OnBeforeRequest” from the dropdown menu (not the “Advanced” one!), and below the REST API the Action will appear:

Select “OnBeforeRequest” for great justice!

There it is, for your modifying pleasures.

As can be seen in the image, the OnBeforeRequest has one Input Parameter, and one Output Parameter. Both are of type HTTPRequest, and contain everything you need to modify the REST message:

The HTTPRequest Structure.

Note that the OnBeforeRequest is a generic way of modifying the REST message, and not specific to multipart/form-data. Also note that there is a single OnBeforeRequest for the entire REST API, not one per Method. This means that inside the OnBeforeRequest, you need to check which Method is called by inspecting the URLPath and HTTPMethod Attributes of the Request Input Parameter².

My solution

As I needed to send binary data, I had to choose the first approach³. This meant I added two Input Parameters: one for the content type, and one for the binary body. Using my Forge Component (see below) I added the multiple parts (including a JSON part) to the message, after which I assigned the concatenated parts via the Binary Data parameter.

Forge Component

Regardless of the specific solution that makes sense for your project, I created a Forge Component, aptly name Multipart/form-data, that consists of a single eSpace with a number of Public Actions that aid in creating the right content:

MultipartFormDataCreate — the main Action, creates a text body consisting of the specified parts, in a multipart/form-data compatible format.

ContentTypeGet — returns the value to be used for the Content-Type header of the REST message based on the boundary used.

PartAdd — a helper Function that can be used to easily create a List of parts.

Wrapping up

That was it, my first article on Medium. I hope you like it, and find it useful when implementing REST with multipart/form-data. Any questions can be asked on the OutSystems Community Forum, the component’s subforum, or as a last resort via PM.