This is the first proper security issue that I have worked on in the wild, so I decided to document my methodology. We were initially tipped off from a quote on this page:

XML-RPC can be used over HTTP sockets, and this is what we do. This comes with a nice side effect: we don't even need to run the separate parts on the same machine, they can be connected remotely over the Internet! … The security-sensitive part, i.e. the renderer, is contained into a relatively simple executable. It's possible to start this executable within a container, so that security issues with the renderer (or, who knows, with the GUI toolkit) won't ever reach beyond the boundaries of the RPC calls.

My personal effort was then focused towards the next core XML-RPC server, as that seemed the most likely to be vulnerable. After a bunch of tweaking, I eventually was able to fire the make.buffers rpc call over a local client, and then over a local js browser, and lastly a remote website, validating the approach.

Finding how to turn this access into RCE was a bit tricky. The commands offered by the core didn't seem very useful at first, mainly due to not fully understanding them. I then pivoted to trying to exploit the platform port, but I quickly found that:

It was unstable, closing whenever any invalid arguments were passed to it. It offered much more in terms of api functionality, but it wouldn't be a good target if we're aiming for RCE due to sandboxing. It might be possible to get XSS or some man in the middle attack, but that wasn't my goal. Snooping on calls from core to the platform port is harder, as in order to do that, you need to kill the platform port (at which point, you wouldn't have any window to control).

I then decided to forcibly kill the core (which seemed to leave the platform port running as if nothing had happened). I then put an echo server on the same port and was able to immediately see all requests being made by the port. After finding out that the port sent keystrokes to the core, and that there was a default command called COMMAND-EVALUATE , the rest was trivial.

In retrospect, I would have liked to have handled disclosure a bit better. It's probably a good idea to give core developers a heads up as soon as you even think there's an issue, to maximize the time they have to fix it. I spent a lot of time sitting on this issue, leaving people vulnerable for longer than they should have been.