Introduction

Imagine you'd like to build your own chat server, which allows clients to exchange messages safely. You have a simple infrastructure consisting of a server written in Ruby and clients for iOS and Android. This is exactly what the famous Mobile websocket example provides. We have modified it to illustrate how simple it is to add security features using Themis.

In this tutorial, we'll try to preserve as much of it's simplicity and architecture as possible, but add cryptographic protection. In case you've randomly bumped into this blog post, Themis is a cross-platform cryptographic framework, providing convenient services to easily store and exchange data over the network.

Architecture overview

Mobile websocket example (MWE) regular workflow consists of two parts:

Sign-in process:

A server listens to websocket, once server receives a new connection, it subscribes a new user to the main chat channel and informs everybody that a new user joins:

Message exchange:

For each client, the server keeps separate event loop in memory, listening to events from this specific client. When somebody pushes something to a server over their connection, server relays that message to everyone via channel everybody is subscribed to.

Adding Themis into scheme

The plan:

Add names for users to identify each other (and for server to identify their keys). Add Secure Session between device and server to provide protection. Add Secure Cell storage on mobile devices to protect stored data.

Secure Session is a lightweight session-based communication protection scheme. It is protocol agnostic and operates on the 5th layer of the network OSI model. Secure Session provides:

secure end-to-end communication

perfect forward secrecy

strong mutual peer authentication

replay protection

low negotiation round-trip

uses strong cryptography (including ECC)

Communications over Secure Session contains 2 stages: negotiation (key agreement) stage and actual data exchange. In our example, client and server will use hardcoded server keys (private and public correspondingly) to start the session. Depending on language you use, pick 'key pair generation' section from language reference of Themis documentation

Secure Cell is simple protected data container, aimed to protect arbitrary data being stored in various types of storages (like databases, filesystem files, document archives, cloud storage etc). It's rather simple, but includes strong cryptographic protection and data authentication mechanisms.



After dropping Themis, the sign-in process will look like this:

Client "newcomer" will perform key agreement procedure and session establishment procedure with the server over websocket, resulting in the generation of a temporary key, which will be used by this client and server to exchange data. This temporary key is bound to session identifier, which will allow the server to pick the right key for each session with each user (more on that in Ruby server section)



Message exchange will look like this:

Every message, which server is able to decrypt (e.g., for which server has an active session) is decrypted with a session key, then encrypted with each session key available to the system and sent to all clients.

Warning: This architecture is used only to show how simple integrating Themis into your application actually is. We preserve as much of initial architecture and process flow as possible. This, however, opens a few problems:

Bandwidth degradation : for N users, every message will be sent N*N times instead on N times (because we send each encrypted message to the same channel and everybody will receive them), so traffic footprint will be N times more than normal.

Attack on session keys: since client knows the message he sent, by getting it encrypted with every other user's key from server, he is able to try "known plaintext attack" (e.g. try to retrieve keys from encrypted text and known plaintext message). Modern ciphers used in Secure Session are safe against this attack, and overall security in our case good. But in general, such architectural patterns introduce risks and should be avoided.

Using the tutorial code

Since we like tutorials which really work, the whole infrastructure is ready to run on localhost. In case you'd like more real-world testing with addresses, which face the internet, you need to change them both in application code and in ruby server code, and make sure addresses and ports are available in the system you're running server code on.

Ruby

Modified Ruby code to see the result of this section is available at corresponding directory in GitHub repo.

Building Ruby Server

First, we will build Ruby server, then add Themis services into it. To do that, we'll install Ruby and Themis (we're using clean Ubuntu installation within typical VM):

apt-get install build-essentials ruby ruby-dev libssl-dev gem install em-websocket gem install rubythemis

Since Ruby is rapidly changing ecosystem, if your rubythemis installation fails for any reason, just refer to current version of ruby guide.



Let's see how the server works in original Mobile websocket example:

require "em-websocket" EventMachine.run do @channel = EM::Channel.new EventMachine::WebSocket.start(host: "0.0.0.0", port: 8080, debug: true) do |ws| ws.onopen do sid = @channel.subscribe { |msg| ws.send(msg) } @channel.push("#{sid} connected") ws.onmessage { |msg| @channel.push("<#{sid}> #{msg}") } ws.onclose { @channel.unsubscribe(sid) } end end end

For each incoming user, we:

subscribe remote user to websocket channel via @channel.subscribe notify everyone that a new user is present create event listener, which reacts to incoming messages over websocket by pushing them into channel when user closes the websocket, we detach him from channel.

This provides basic chat server experience in our case.

Adding Themis cryptography to Ruby server

Now, let's add all the components necessary to provide encrypted Secure Session services within this chat server:

add dependencies add server key create infrastructure for storing the session keys: for every new session we create ephemeral keys, which are valid only for this session. since the server will handle a lot of sessions simultaneously, Themis library needs some way to know which key to use with current session. Callback is simple and appropriate pattern here. instead of just receiving the message and pushing it to everyone, we use Secure Session's state to control flow after receiving any message via websocket - each message can be treated as:

signal to initiate new session: user sends his key and id, we store it and consider this as user's signal to start key acknowledgment.

ongoing session initiation traffic (key negotiation takes a few steps, which Secure Session object performs itself)

when session is established, message is expected to contain Secure Session message within it, which decrypts to regular text.

First, we'll add dependences (we'll need base64 to send binary data in strings):

require "rubythemis" require "base64"

Then, we need to add pre-shared server private key:

server_priv_key = "\x52\x45\x43\x32\x00\x00\x00\x2d\x49\x87\x04\x6b\x00\xf2\x06\x07\x7d\xc7\x1c\x59\xa1\x8f\x39\xfc\x94\x81\x3f\x9e\xc5\xba\x70\x6f\x93\x08\x8d\xe3\x85\x82\x5b\xf8\x3f\xc6\x9f\x0b\xdf"

We'll need two arrays - one for keys and one for sessions, and a callback method to return user's key according to it's id. Callback is required by Secure Session architecture to be able to access keys while establishing the session:

$pub_keys = Hash.new $sessions = Hash.new callback method class Callbacks_for_themis < Themis::Callbacks def get_pub_key_by_id(id) return $pub_keys[id].force_encoding("BINARY") end end

Then we start EventMachine with websocket listener:

EventMachine.run do @channel = EM::Channel.new EventMachine::WebSocket.start(host: "0.0.0.0", port: 8080, debug: true) do |ws| ws.onopen do

Secure Session is established via sequence of stages, and in each moment in time can be in three states:

none : (stage 0) session between two parties is not present

initiating : (stage !0, result = 1) session is being initiated right now

initiated: (stage !0, result > 1) session is initiated and ready to use

Based on session stage, we treat incoming messages in a different manner:

case 1 : if stage is 0, we create new session identifier and allocate variables, callbacks and various tooling

case 2 : if stage is not 0, we try to decrypt a message and based on result we:

case 2-1 : if result of decrypting is 1, we're still in progress of session initiation, so we just have to carry on with it



case 2-2: if result is > 1 and stage is not 0, this is a sign of established session and we can execute normal flow: for each session, encrypt message with it's session key and push into main channel.

Case 1: if Stage is 0:

id_pubkey = msg.split(":") id=id_pubkey[0] $pub_keys[id_pubkey[0]]=Base64.decode64(id_pubkey[1]) callbacks = Callbacks_for_themis.new ssession = Themis::Ssession.new("server", server_priv_key, callbacks) $sessions[id]=ssession stage+=1

Case 2: Stage is not 0: try to decrypt a message and retrieve control information from Themis:

res, mes = $sessions[id].unwrap(Base64.decode64(msg)) # Trying to decrypt message

Case 2-1: Is decryption result 1?: if it's 1, carry on establishing the session

if res == 1 #if res equals Themis::SEND_AS_IS, we're still initiating the session ws.send(Base64.encode64(mes))

Case 2-2: If decryption result is not 1: it means the session is established and we can get messages from the client, decrypt them and forward to other users, encrypting it with each user's keys. This way, we send channel-wide messages only when we have the message coming over established session:

$sessions.each do |sid, session| @channel.push(Base64.encode64($sessions[id].wrap(mes)))

Voila! We've replaced handling of incoming messages by pushing them to the channel (using all the possible states of Secure Session in the process).

When client unbinds, we delete session data from sessions hash and delete public key.

ws.onclose { @channel.unsubscribe(sid) $sessions.delete(id) $pub_keys.delete(id)

That's it! By replacing message forwarding handler with simple control flow, you now have Secure Session protecting your messages for you.

But what about the clients? Since it's yet impossible to run Themis in your web browser (however, this will change sooner than you would have imagined), our next targets will be two mobile platforms.

Android

Setting up development environment

Once again, we're using a clean Ubuntu install.

If you follow google developer instructions properly, everything will go smoothly. However, keep one thing in mind - if you have a fresh default Ubuntu machine (tested on 14.04 LTS) you should install zlib1g apart from JDK described in Android docs:

sudo apt-get install zlib1g

Or on x64:

sudo apt-get install zlib1g:i386

If you do not do this and try to run any build from Android Studio, it will freeze forever without giving you any errors or reasons.

Building Themis for Android

By following Themis installation guide, you should get working Themis installation.

However, if Themis progresses far enough from time this tutorial is being written, you might want to use it in state, in which this tutorial was written with:

git clone https://github.com/cossacklabs/themis cd themis git reset --hard b6e8e4d479d2fa6118423059021d864f1ef763e2

Adding Themis to chat app

The Android client for mobile websocket example (MWE) is easily opened and built by recent Android Studio.

First, we've added a reference to Themis Android library to the client. There is one slight 'hack', which is MWE's AndroidManifest.xml: the MWE client declares minimal suported SDK version as 14 (Android 4.0), but Themis prefers 16 (Android 4.1), so we need to update the MWE client to comply. Why bother trying to add security for old buggy and breachy Android releases in the first place?

So, build.gradle:

minSdkVersion 14 -> minSdkVersion 16

And AndroidManifest.xml:

android:minSdkVersion="14" -> android:minSdkVersion="16"

The plan:

Add Themis infrastructure into the app, generate keys on startup Patch all client-server functions to talk through Secure Session Extend the example to save message history between app cycle securely with Secure Cell

And to the fun part - MainActivity.java:

Importing Themis functions:

import com.cossacklabs.themis.ISessionCallbacks; import com.cossacklabs.themis.InvalidArgumentException; import com.cossacklabs.themis.KeyGenerationException; import com.cossacklabs.themis.KeypairGenerator; import com.cossacklabs.themis.Keypair; import com.cossacklabs.themis.NullArgumentException; import com.cossacklabs.themis.PublicKey; import com.cossacklabs.themis.SecureCellData; import com.cossacklabs.themis.SecureCellException; import com.cossacklabs.themis.SecureSession; import com.cossacklabs.themis.SecureSessionException; import com.cossacklabs.themis.SecureCell;

Adding some new private members to the activity:

// This will be our keypair to communicate with the server private Keypair keypair; // Secure session for our WebSocket private SecureSession secureSession; // DeviceID: we will use it as a key to protect message history (example only, do not use in production) private final String deviceId = Build.MANUFACTURER + Build.MODEL;

Each time application starts, we'll generate new keypair:

@Override protected void onCreate(Bundle savedInstanceState) { ... keypair = KeypairGenerator.generateKeypair(); ... }

Establish Secure Session with server

To create Secure Session, when connecting through WebSocket, we do the following:

private void connectWebSocket() { ... secureSession = new SecureSession( deviceId.getBytes("UTF-8"), keypair.getPrivateKey(), new ISessionCallbacks() { @Override public PublicKey getPublicKeyForId(SecureSession secureSession, byte[] bytes) { // Assuming we speak to one server only, just returning his "known" private key return new PublicKey(new byte[]{0x55, 0x45, 0x43, 0x32, 0x00, 0x00, 0x00, 0x2d, 0x75, 0x58, 0x33, (byte)0xd4, 0x02, 0x12, (byte)0xdf, 0x1f, (byte)0xe9, (byte)0xea, 0x48, 0x11, (byte)0xe1, (byte)0xf9, 0x71, (byte)0x8e, 0x24, 0x11, (byte)0xcb, (byte)0xfd, (byte)0xc0, (byte)0xa3, 0x6e, (byte)0xd6, (byte)0xac, (byte)0x88, (byte)0xb6, 0x44, (byte)0xc2, (byte)0x9a, 0x24, (byte)0x84, (byte)0xee, 0x50, 0x4c, 0x3e, (byte)0xa0}); } @Override public void stateChanged(final SecureSession secureSession) { // Let's track secure session establishment by adding state notifications to overall message history runOnUiThread(new Runnable() { @Override public void run() { TextView textView = (TextView)findViewById(R.id.messages); textView.setText(textView.getText() + "

" + secureSession.getState().name()); } }); } } ); ... }

Sending the key to server

For the server to know our public key, we need to get introduced first, so we need send it to the server first (see scheme 1 in 'Adding Themis to the scheme'):

Instead of basic insecure "Hello"

:

@Override public void onOpen(ServerHandshake serverHandshake) { ... mWebSocketClient.send("Hello from " + Build.MANUFACTURER + " " + Build.MODEL); ... }

We will send our id + public key:

@Override public void onOpen(ServerHandshake serverHandshake) { ... mWebSocketClient.send(deviceId + ":" + Base64.encodeToString(keypair.getPublicKey().toByteArray(), Base64.DEFAULT)); mWebSocketClient.send(Base64.encodeToString(secureSession.generateConnectRequest(), Base64.DEFAULT)); ... }

Receiving protected messages via Secure Session

As for received messages, instead of regular handler, which just displays incoming messages:

@Override public void onMessage(String s) { final String message = s; runOnUiThread(new Runnable() { @Override public void run() { TextView textView = (TextView)findViewById(R.id.messages); textView.setText(textView.getText() + "

" + message); } }); }

We'll do Secure Session state-based flow:

@Override public void onMessage(String s) { byte[] wrappedData = Base64.decode(s, Base64.DEFAULT); SecureSession.UnwrapResult unwrapResult = secureSession.unwrap(wrappedData); switch (unwrapResult.getDataType()) { case PROTOCOL_DATA: // The session is not established yet. Send response to the server. mWebSocketClient.send(Base64.encodeToString(unwrapResult.getData(), Base64.DEFAULT)); break; case NO_DATA: // Nothing to do... Nothing to send... break; case USER_DATA: // This is our received message. Display it to the user. final String message = new String(unwrapResult.getData(), "UTF-8"); runOnUiThread(new Runnable() { @Override public void run() { TextView textView = (TextView)findViewById(R.id.messages); textView.setText(textView.getText() + "

" + message); } }); break; } }

Sending protected messages via Secure Session

Initial code for sending messages looked like this:

public void sendMessage(View view) { EditText editText = (EditText)findViewById(R.id.message); mWebSocketClient.send(editText.getText().toString()); editText.setText(""); }

We will replace it with encryption part:

public void sendMessage(View view) { EditText editText = (EditText)findViewById(R.id.message); byte[] wrapped = secureSession.wrap(editText.getText().toString().getBytes("UTF-8")); mWebSocketClient.send(Base64.encodeToString(wrapped, Base64.DEFAULT)); editText.setText(""); }

Adding secure storage for message history

Purely for illustrational purposes, we've added local protected storage for anything you'd like to keep safe within your app. In this case, it's history.

It's rather simplistic:

on application startup we try to locate object 'history' in SharedPrefs, and if it's there - expect it to be Secure Cell object, decrypt it and show in text window.

on application shutdown we just record state of textView into variable, which is then encrypted into SecureCell and put into SharedPrefs.

@Override protected void onStart() { super.onStart(); TextView textView = (TextView)findViewById(R.id.messages); SharedPreferences sharedPref = getPreferences(Context.MODE_PRIVATE); if (sharedPref.contains("history")) { // DeviceID is used here just for simplicity of the example. In real-world it should be a "real" password. For example, just prompt the user to enter his storage password when app starts. SecureCell secureCell = new SecureCell(deviceId); SecureCellData protectedData = new SecureCellData(Base64.decode(sharedPref.getString("history", ""), Base64.DEFAULT), null); textView.setText(new String(secureCell.unprotect(deviceId.getBytes("UTF-8"), protectedData), "UTF-8")); } } @Override protected void onStop() { TextView textView = (TextView)findViewById(R.id.messages); SharedPreferences sharedPref = getPreferences(Context.MODE_PRIVATE); SharedPreferences.Editor editor = sharedPref.edit(); SecureCell secureCell = new SecureCell(deviceId); SecureCellData protectedData = secureCell.protect(deviceId.getBytes("UTF-8"), textView.getText().toString().getBytes("UTF-8")); editor.putString("history", Base64.encodeToString(protectedData.getProtectedData(), Base64.DEFAULT)); editor.commit(); super.onStop(); }

iOS

Building and running

To build iOS application you need OSX 10.9+ and Xcode. You can download it from AppStore. Example project was built on Xcode6 using iOS8 SDK, but updated it to support Xcode7 and iOS9 SDK too.

To build the application, please follow the guide. In few words, you need to install CocoaPods, then run ruby server and the run application itself.

The plan

Websocket example application provides simple chatting with Ruby server above. We will extend MWE's iOS client to work with Themis's Secure Session and Secure Cell.

In order to use Themis, we will:

Add Themis lib into project. Generate keys when application starts. Wrap client-server interactions with SecureSession. Store message history with SecureCell.

Adding Themis

If you inspect Podfile, you will definitely find there two pods: SocketRocket and latest Themis.

You can link Themis on head of master branch, on specific commit or on latest release:

`pod "themis", :git => 'https://github.com/cossacklabs/themis.git', :commit => 'b6e8e4d479d2fa6118423059021d864f1ef763e2'`

If you prefer not to use CocoaPods, just drag sources into your project and add `OpenSSL-Universal` lib.

Initialise keys

Server public key is hardcoded into the app. Usually, hardcoding keys is a bad practice, but good enough for demonstration purposes.

NSString * const kServerKey = @"VUVDMgAAAC11WDPUAhLfH+nqSBHh+XGOJBHL/cCjbtasiLZEwpokhO5QTD6g";

Next, we need to generate private and public key:

TSKeyGen * keygenEC = [[TSKeyGen alloc] initWithAlgorithm:TSKeyGenAsymmetricAlgorithmEC]; if (!keygenEC) { [self loggingEvent:@"Error while initializing keygen"]; return; } NSData * privateKey = keygenEC.privateKey; NSData * publicKey = keygenEC.publicKey;

Connecting to server

In order to connect with localhost server, you need to run ruby server from the terminal. If server is not running yet, please, follow guide mentioned above :)

- (void)connectWebSocket { NSString * urlString = @"ws://127.0.0.1:8080"; self.webSocket = [[SRWebSocket alloc] initWithURL:[NSURL URLWithString:urlString]]; self.webSocket.delegate = self; [self.webSocket open]; }

Sending key to server, aka handshake messages

As previously agreed, the first message from the client to server is handshake in format `device name:public key'.

NSString * name = [UIDevice currentDevice].name; NSString * handshakeMessage = [NSString stringWithFormat:@"%@:%@", name, [publicKey base64EncodedStringWithOptions:0]]; [self.webSocket send:handshakeMessage];

Establishing the session

Establish session, initialize Transport object and TSSession object using client's private key.

// send establishment message self.transport = [Transport new]; self.session = [[TSSession alloc] initWithUserId:[name dataUsingEncoding:NSUTF8StringEncoding] privateKey:privateKey callbacks:self.transport]; NSError * error = nil; NSData * sessionEstablishingData = [self.session connectRequest:&error]; if (error) { [self loggingEvent:[NSString stringWithFormat:@"Error while handshake %@", error]]; return; } NSString * sessionEstablishingString = [sessionEstablishingData base64EncodedStringWithOptions:0]; [self.webSocket send:sessionEstablishingString];

If the session is not established yet, client unwraps response from the server and sends it back:

- (void)webSocket:(SRWebSocket *)webSocket didReceiveMessage:(id)message { ... NSData * receivedData = [[NSData alloc] initWithBase64EncodedString:message options:NSDataBase64DecodingIgnoreUnknownCharacters]; NSData * unwrappedMessage = [self.session unwrapData:receivedData error:&error]; if (![self.session isSessionEstablished] && unwrappedMessage) { NSString * unwrappedStringMessage = [unwrappedMessage base64EncodedStringWithOptions:0]; [webSocket send:unwrappedStringMessage]; return; } ... }

Sending protected user messages via SecureSession

After these simple initialization steps are done, it's time to send user's messages to the server. But first, we should wrap and encode each message to NSData, and send it as base64 NSString.

- (IBAction)sendMessage:(id)sender { NSData * dataToSend = [self.messageTextField.text dataUsingEncoding:NSUTF8StringEncoding]; // wrap data NSError * error; NSData * wrappedData = [self.session wrapData:dataToSend error:&error]; if (!wrappedData || error) { [self loggingEvent:[NSString stringWithFormat:@"Error on wrapping message %@", error]]; return; } NSString * wrappedStringMessage = [wrappedData base64EncodedStringWithOptions:0]; [self.webSocket send:wrappedStringMessage]; }

Receiving protected server messages via SecureSession

Protected messages are encoded, so it's not enough just to read them from socket. First, message is unwraped to NSData, checked on errors, then transformed into NSString.

- (void)webSocket:(SRWebSocket *)webSocket didReceiveMessage:(id)message { ... NSData * receivedData = [[NSData alloc] initWithBase64EncodedString:message options:NSDataBase64DecodingIgnoreUnknownCharacters]; NSData * unwrappedMessage = [self.session unwrapData:receivedData error:&error]; ... NSString * unwrappedString = [[NSString alloc] initWithData:unwrappedMessage encoding:NSUTF8StringEncoding]; }

Storing history

Storing chat history may be valuable feature for users, and is a nice use-case to illustrate protected local storage.

The flow is trivial:

Create SecureCell object that handles encrypting/decrypting messages. Encrypt and put every message to storage. Read history and decrypt messages on application start.

Let's create SecureCell object and save it to `self.secureStorageEnryptor` property.

- (void)createSecureStorage { // create encryptor if there's no if (!self.secureStorageEnryptor) { // you should NEVER use uuid as encryption key ;) NSString * encryptionKey = [[[UIDevice currentDevice] identifierForVendor] UUIDString]; self.secureStorageEnryptor = [[TSCellSeal alloc] initWithKey:[encryptionKey dataUsingEncoding:NSUTF8StringEncoding]]; } }

Put each message to history when user sends message or receives server's response. Ask SecureCell to encrypt messages and store them in NSUserDefaults.

- (void)saveHistory:(NSString *)string { // create encryptor if there's no [self createSecureStorage]; // encrypt event NSError * error = nil; NSString * message = [NSString stringWithFormat:@"%@ %@", [NSDate date], string]; NSData * encryptedEvent = [self.secureStorageEnryptor wrapData:[message dataUsingEncoding:NSUTF8StringEncoding] context:nil error:&error]; if (!encryptedEvent || error) { NSLog(@"Error on encrypting message %@", error); return; } NSUserDefaults * userDefaults = [NSUserDefaults standardUserDefaults]; // check if history is already presented NSMutableArray * history = [[userDefaults valueForKey:kHistoryStoringKey] mutableCopy]; if (!history) { history = [NSMutableArray new]; } // add encrypted object to history [history addObject:encryptedEvent]; // add history to storage [userDefaults setObject:history forKey:kHistoryStoringKey]; }

Reading history data, decrypting every message.

- (void)readHistory { NSLog(@"Previous message history on this client..."); [self createSecureStorage]; NSUserDefaults * userDefaults = [NSUserDefaults standardUserDefaults]; NSMutableArray * history = [userDefaults valueForKey:kHistoryStoringKey]; [history enumerateObjectsUsingBlock:^(NSData * encryptedData, NSUInteger idx, BOOL * stop) { NSError * error = nil; NSData * decryptedMessage = [self.secureStorageEnryptor unwrapData:encryptedData context:nil error:&error]; if (error) { NSLog(@"Error on decrypting message %@", error); } else { NSString * resultString = [[NSString alloc] initWithData:decryptedMessage encoding:NSUTF8StringEncoding]; NSLog(@"Message decrypted:

-- %@", resultString); } }]; NSLog(@"End of history



");

Enjoy encrypted chatting and feel free to contribute!

2018 UPD: This article is still valid, yet Themis has significantly evolved since this article was posted.

If you're looking for new ideas, this is the right place. If you're looking to implement security, apply for our Customer Success Program. If you're looking for ready-made solutions, consider looking into Themis, Acra, Hermes, or Toughbase.