Creating an Observable

The jsm binary allows to create an Observable in an interactive way. Several parameters need to be set:

the name of the underlying Message Set

the Durable Name defines the name of the durable subscription (durable is a concept introduced in NATS Streaming). While JetStream supports durable and ephemeral Observables (this last one requiring an active subscriber), in the current version only durables are supported by jsm

defines the name of the durable subscription (durable is a concept introduced in NATS Streaming). While JetStream supports durable and ephemeral Observables (this last one requiring an active subscriber), in the current version only durables are supported by jsm the type of consumption: Push vs Pull based.

the subject consumers should subscribed to in order to get the messages (only for Push)

the Delivery Policy defines how the messages will be delivered. Several possibilities here: delivery since the first message, only the last message, all messages since a specified time

defines how the messages will be delivered. Several possibilities here: delivery since the first message, only the last message, all messages since a specified time the Acknowledgement Policy defines if and how a client application should acknowledge the message delivery: none (NoAck) means no ack is expected, all (AckAll) means all messages until the current sequence number will be acknowledged, AckExplicit means each message needs to be acknowledge individually

defines if and how a client application should acknowledge the message delivery: none (NoAck) means no ack is expected, all (AckAll) means all messages until the current sequence number will be acknowledged, AckExplicit means each message needs to be acknowledge individually the Replay Policy defines the way the messages are delivered. Two available options: all messages in a row without any specific delay between each of them, or one by one respecting the original delay existing between each of them

In this example, we configured the Observable with the following parameters:

the underlying Message Set is devices-events

the durable name is set to temperature (as it is used to get temperature related messages from the devices)

(as it is used to get temperature related messages from the devices) it is defined as a Push based consumption

based consumption messages will be pushed to all consumers which subscribed to the devices.events subject. Messages will be delivered using the original subject on which they were published (more on that in a bit).

Note: from a scaling perspective, it is not recommended to have multiple consumers for Push based Observable. Pull based Observable allow easy scaling.

the default values are used for all the other options

$ jsm add-obs

Enter the following information

Message Set Name: devices-events

Durable Name: temperature

Push or Pull: Push

Delivery Subject: devices.events

Deliver All? (Y|n):

AckPolicy (None|all|explicit):

Replay Policy (Instant|original):

Received response of "+OK"

Using jsm, we can list all the Observables for a given Message:

$ jsm ls-obs devices-events 1) temperature

We can also get the information of the Observable, using its name and the Message Set it operates on:

$ jsm info-obs devices-events temperature

Received response of {

"Delivered": {

"ObsSeq": 1,

"SetSeq": 1

},

"AckFloor": {

"ObsSeq": 0,

"SetSeq": 0

},

"Pending": null,

"Redelivery": null

}

Several things to note here:

the next message to be delivered is the one with sequence number 1

no messages have been acknowledged yet

no messages are currently pending (delivered but not acknowledged)

no messages have been redelivered

Now the Observable is created a regular NATS client can subscribe to the “devices.events” subject in order to receive the messages stored within the underlying Message Set.

Let’s consider a micro-service, named listener, in charge of collecting and parsing each message and storing its content into a time series database (InfluxDB could be a great candidate for this purpose but this is another story). The following Python code is a simplified version of the listener:

import asyncio

import json

import ssl

import sys

from nats.aio.client import Client as NATS async def run(loop): # Connection to NATS

nc = NATS()

await nc.connect(io_loop=loop) # Handling incoming messages

async def message_handler(msg):

data = json.loads(msg.data.decode())

print(data) # Subscription to "devices.events"

subject="devices.events"

await nc.subscribe(subject, cb=message_handler)

print('Subscribed to [%s]' % subject); if __name__ == '__main__':

loop = asyncio.get_event_loop()

loop.run_until_complete(run(loop))

try:

loop.run_forever()

finally:

loop.close()

An important thing to note here: the listener subscribes to the “devices.events” subject while each device sends messages to the “devices.ID.events” subject (ID being the unique identifier of a device). Indeed, event if the listener subscribes to “devices.events”, it will receive messages on the message’s original subject. This behavior might be changed in a next release so that clients can directly subscribe to the original subject.

Starting the listener, we can see all the messages are delivered starting for the first one in the Message Set. The following animation illustrates all the messages are received by the listener. The status of the Observable is changed showing all the messaging have been delivered (there were 25 messages in the Message Set in this example)

Subscriber receiving messages stored in a Message Set

If additional messages are published by the devices, they will be received by the listener. Also, if the listener happens to be offline, while devices keep on publishing messages, it will receive messages starting from the last sequence number it acknowledged.