\ Get the ZeroMQ interface:

needs net/zmq



\ The zmq context:

var ctx



\ The zmq subscription socket:

var sub



\ Connection setup:

: init

zmq:ctx-new ctx !

ctx @ zmq:SUB zmq:socket dup sub !



sub @ "tcp://localhost:5557" zmq:connect drop

sub @ "Topic" zmq:subscribe ;



zmq:msg_t var, msg



: app:main

init

"Listening for large messages..." . cr

repeat

sub @ msg @ zmq:getmsg[]

\ Get the last of the series of messages and drop the message array:

-1 a:@ nip

\ If it's a 'big' message, write it out to a log file for dissection:

b:len 10000 n:> if

"/tmp/msg.raw" f:create swap f:write drop f:close

"Wrote a message to the log..." . cr

else

\ small messages are just ignored...

drop

then

again ;

I've taken over development of a fairly complex code-base for one of my clients. The project involves, among other things, a web-server (in nodejs) which is the interface to the user, and a back-end server (in C++) which handles the actual work. This is, of course, a quite typical application architecture.The communications between the web side and the work side were done using a messaging library which, though good, was not quite right for this system's architecture. So after doing research, I hit upon using "ZeroMQ" , and in 8th 16.14 I added an interface library in 8th to it.To make the transition from the other library to ZeroMQ easier, i decided to change things as little as possible, keeping the same semantics as the other library. This was only a little challenging. What was a bigger challenge was when I ran the code, I noticed that most of the time it worked as expected, but sometimes -- particularly on big requests -- it didn't work.There are a number of "moving parts" in the system, and I was unsure where the problem was. So I decided I should first of all "sniff" the line to see if the data being sent from the web side matched my expectations. This code is what I used to debug the line trace:(Note: this code requires some extra functionality in the 'net/zmq' library which will appear in the next 8th release)By changing "Topic" to be the topic to which I wanted to listen, I was able to determine quickly that the data sent from the web side matched my expectations precisely. However, the data sent on the work side to the workers did not. Further investigation showed that the "zmq_msg_send" which relayed the request to the workers did have the correct information, but after sending, the data were corrupted.This made me examine the ZeroMQ documentation more closely, where I found that it is "zero copy" and thus the data being sent must remain in-scope throughout the call-cycle. When I looked at the worker code I noticed a local-variable held the data to be retransmitted. By simply making that variable heap-based, I was able to fix the problem (and by using the ZeroMQ's library properly, I was also able to avoid memory leaks).In this story, 8th was the hero because it let me quickly debug the line-protocol and determine where the point of failure in the main code must be.