Salt Stack XMPP Gateway

Salt is very open ended. That is one attribute I love in the software I am using. One dictionary defines open ended like this:

Not restrained by definite limits, restrictions, or structure. Allowing for or adaptable to change. Inconclusive or indefinite Allowing for a spontaneous, unstructured response: an open-ended question.

Except for that third point, that is pretty much the Salt Stack definition right there. Sorry for sounding like a salesman, but after using under 2 hours to build a simple XMPP gateway to control possibly thousands of minions, allow me to feel a little giddy.

Let me demonstrate, here we go!

Prosody, a lightweight XMPP-server in Lua

I have used Prosody in a couple of projects earlier, it's very light weight and easy to work with, it will serve perfectly for this proof of concept.

Install prosody

apt-get install prosody

Configure prosody

/etc/prosody/prosody.cfg.lua

admins = { 'admin@salt.demo.no' } "watchregistrations"; -- Alert admins of registrations allow_registration = true; VirtualHost "salt.demo.no"

Restart prosody

service prosody restart

This demo uses two XMPP-users, one admin-account that will control the other account, used by the gateway.

Add admin account, set a password

prosodyctl adduser admin@salt.demo.no

Add salt account, set a passord

prosodyctl adduser salt@salt.demo.no

At this point you can log in with your XMPP client to admin@salt.demo.no to see if XMPP server is working.

Salt API

Salt API is a REST API for Salt, allowing you to interface with Salt over HTTP. For this demo I could have used python local-client directly. But using Salt API allows for separation of XMPP and salt, say if you want to build agents for thousands for minions, or just have nice separation.

Configure salt api (check Salt API docs for details)

/etc/salt/master

rest_cherrypy: port: 8000 ssl_crt: /etc/pki/tls/certs/localhost.crt ssl_key: /etc/pki/tls/certs/localhost.key debug: True external_auth: pam: xmppuser: - test.* - status.*

Start salt-api

salt-api

We allow the XMPP-gateway user to use two salt modules: test and status (e.g. test.version, status.uptime)

Add the user xmppuser:

adduser xmppuser

Install XMPP library for python

apt-get install python-xmpp

The Salt XMPP gateway script

Note: This is just a demo, and not meant for production usage. The gateway script fires up a master bot and 1 minion bot for every minion. It will start off by trying to register a new account in-band for every minion and then send a message to the specified admin reporting in that it's ready for duty. The gateway has one command minions to list all the minions. All other text is treated as a function, e.g. test.version If you talk to a minion, it will run the command for that minion, if you talk to the master, it will run command on all minions. I have no idea on how this scales, as python will start a thread for each minion, eventually this will all break. My tests are for a small set of minions.

Screenshots below!

Installation

First adapt my configuration into your needs and save it to the file config.yaml:

config.yaml

saltapiurl: 'http://localhost:8000/' saltuser: 'xmppuser' saltpass: '73Hengebruer' xmppadminuser: 'admin@salt.demo.no' stripdomain: '.demo.no' username: 'salt@salt.demo.no' password: '66Kjensleuttrykket'

Then I wrote a simple Salt REST API python client. It's inspired by the pepper utility that will eventually do that, but it's not finished yet.

saltrest.py

import urllib import urllib2 from cookielib import CookieJar import json HEADERS = { 'Accept': 'application/json', 'Content-Type': 'application/json', 'X-Requested-With': 'XMLHttpRequest', } cj = CookieJar() class MyHTTPRedirectHandler(urllib2.HTTPRedirectHandler): def http_error_302(self, req, fp, code, msg, headers): # patch headers to include salt token token = headers['X-Auth-Token'] HEADERS['X-Auth-Token'] = token return urllib2.HTTPRedirectHandler.http_error_302(self, req, fp, code, msg, headers) http_error_301 = http_error_303 = http_error_307 = http_error_302 cookieprocessor = urllib2.HTTPCookieProcessor(cj) opener = urllib2.build_opener(MyHTTPRedirectHandler, cookieprocessor) urllib2.install_opener(opener) class SaltREST(object): def __init__(self, config): self.config = config self.login() def login(self): lowstate_login =[{ 'eauth': 'pam', 'username': self.config['saltuser'], 'password': self.config['saltpass'], }] postdata = json.dumps(lowstate_login).encode() req = urllib2.Request(self.config['saltapiurl']+'login', postdata, HEADERS) f = urllib2.urlopen(req) return f.read() def get_minions(self): lowstate = [{ 'client': 'local', 'tgt': '*', 'fun': 'test.version', }] postdata = json.dumps(lowstate).encode() req = urllib2.Request(self.config['saltapiurl']+'', postdata, HEADERS) f = urllib2.urlopen(req) ret = json.loads(f.read()) # Format minions as a list with minion FQDNs ret = [x.replace(self.config['stripdomain'], '') for x in ret['return'][0].keys()] return ret def call(self, lowstate): postdata = json.dumps(lowstate).encode() req = urllib2.Request(self.config['saltapiurl'], postdata, HEADERS) f = urllib2.urlopen(req) ret = json.loads(f.read()) return ret

This is the botherder script:

salt-xmpp.py

import sys import os import xmpp from threading import Thread, Event import yaml # Our Salt REST API import saltrest # flag to tell all threads to stop _stop = Event() def single_node_xmpp_outputter(ret): ret = ret['return'][0] fret = '' for host, val in ret.items(): fret += '%s

' %val return fret def xmpp_outputter(ret): ret = ret['return'][0] fret = '' for host, val in ret.items(): fret += '%s: %s

' %(host.replace(CONFIG['stripdomain'], ''), val) return fret def masterMessageCB(conn, mess): text=mess.getBody() user=mess.getFrom() jid = xmpp.protocol.JID(user).getStripped() print 'Got command:', text if jid == CONFIG['xmppadminuser']: if text == 'minions': # make a nice list conn.send(xmpp.Message(mess.getFrom(), ', '.join(MINIONS))) else: lowstate = [{ 'client': 'local', 'tgt': '*', 'fun': text, }] ret = xmpp_outputter(salt.call(lowstate)) conn.send(xmpp.Message(mess.getFrom(), ret) ) def make_msg_handler(tgt): def minionCB(dispatcher, mess): print '[%s] %s' % (dispatcher._owner.Resource, mess) text=mess.getBody() user=mess.getFrom() jid = xmpp.protocol.JID(user).getStripped() if jid == CONFIG['xmppadminuser']: lowstate = [{ 'client': 'local', 'tgt': tgt + CONFIG['stripdomain'], 'fun': text, }] ret = single_node_xmpp_outputter(salt.call(lowstate)) dispatcher.send(xmpp.Message(mess.getFrom(), ret) ) return minionCB def startminion(username, password): jid=xmpp.protocol.JID(username) cli=xmpp.Client(jid.getDomain(), debug=False) cli.connect() should_register = True if should_register: # getRegInfo has a bug that puts the username as a direct child of the # IQ, instead of inside the query element. The below will work, but # won't return an error when the user is known, however the register # call will return the error. xmpp.features.getRegInfo(cli, jid.getDomain(), #{'username':jid.getNode()}, sync=True) if xmpp.features.register(cli, jid.getDomain(), {'username':jid.getNode(), 'password':password}): sys.stderr.write("Successfully register: %s!

" %jid.getNode()) else: sys.stderr.write("Error while registering: %s

" %jid.getNode()) authres=cli.auth(jid.getNode(),password) if not authres: print "Unable to authorize %s - check login/password." %jid.getNode() return None #sys.exit(1) if authres<>'sasl': print "Warning: unable to perform SASL auth. Old authentication method used!" cli.RegisterHandler('message', make_msg_handler(jid.getNode())) cli.sendInitPresence() cli.send(xmpp.protocol.Message(CONFIG['xmppadminuser'],'Hello, Salt minion %s reporting for duty.' %jid.getNode())) return cli def startmaster(username, password): jid=xmpp.protocol.JID(username) cli=xmpp.Client(jid.getDomain(), debug=False) cli.connect() authres=cli.auth(jid.getNode(),password) if not authres: print "Unable to authorize - check login/password." sys.exit(1) if authres<>'sasl': print "Warning: unable to perform SASL auth. Old authentication method used!" cli.RegisterHandler('message', masterMessageCB) cli.sendInitPresence() cli.send(xmpp.protocol.Message(CONFIG['xmppadminuser'],'Salt gateway ready for action.')) return cli def process_until_disconnect(bot): ret = -1 while ret != 0 and not _stop.is_set(): ret = bot.Process(1) root = os.path.dirname(os.path.abspath(__file__)) CONFIG = yaml.safe_load(file(root+'/config.yaml').read()) salt = saltrest.SaltREST(CONFIG) # Get minions so we can create bots, uses test.ping to get minion list MINIONS = salt.get_minions() username = CONFIG['username'] password = CONFIG['password'] _stop.clear() # Start master masterbot = startmaster(username, password) try: Thread(target=process_until_disconnect, args=(masterbot,)).start() for minion in MINIONS: minionbot = startminion(minion+'@salt.idrift.no', 'sharedbotpwfordemo') if minionbot: Thread(target=process_until_disconnect, args=(minionbot,)).start() # Block main thread waiting for KeyboardInterrupt #while True: # pass except KeyboardInterrupt: _stop.set() print "Bye!"

Jitsi client showing the chat session with the Salt XMPP gateway

Jitsi client showing the chat session with a salt minion

Source: https://github.com/torhve/salt-xmpp