Keywhiz

Keywhiz is a system for managing and distributing secrets. It can fit well with a service oriented architecture (SOA). Here is an overview in presentation format

Every organization has services or systems that require secrets. Secrets like:

TLS certificates/keys

GPG keys

API tokens

database credentials

Common practices include putting secrets in config files next to code or copying files to servers out-of-band. The former is likely to be leaked and the latter difficult to track.

Keywhiz makes managing secrets easier and more secure. Keywhiz servers in a cluster centrally store secrets encrypted in a database. Clients use mutually authenticated TLS (mTLS) to retrieve secrets they have access to. Authenticated users administer Keywhiz via CLI. To enable workflows, Keywhiz has automation APIs over mTLS.

Keywhiz is reliable and used in production, however occasional changes may break API backward compatibility.

System components

Keywhiz Server

Keywhiz Server provides JSON APIs for accessing and managing secrets. It is written in Java and based on Dropwizard.

Keysync

Keysync is a Keywhiz client that retrieves secrets from a Keywhiz Server using mTLS with a client certificate and stores them in tmpfs.

Presenting secrets as files makes Keywhiz compatible with nearly all software. Outside of Keywhiz administration, consumers of secrets only have to know how to read a file.

Keysync stores all secrets in memory only (via tmpfs) and never persists them to disk. If the server loses power, all secrets will be safely removed from that server.

Keysync's design mitigates a Keywhiz Server outage, since all necessary secrets are stored locally, and can be accessed even if Keysync process is restarted. The full sync of secrets is only necessary in the event of a server reboot.

Keywhiz CLI

Keywhiz CLI is a Java program for Keywhiz administration. Clients, secrets, and groups can be queried, added, removed, or associated with each other. Users can authenticate and use the CLI.

Public Key Infrastructure

Keywhiz makes heavy use of mTLS and X509 certificates. It can even help distribute and rotate them for other services! There is the assumption of a PKI system though. If one does not exist or a PKI is wanted for development consider certstrap for a simple, initial PKI.

Data model

The data model is composed of clients, secrets, groups, and users.

Clients are identified by a client certificate. A database entry can enable/disable access or escalate automation privileges.

Secrets have a globally unique name and can contain any binary content. For example, helloworldsrv-database.yml . Secrets are immutable, but multiple ordered versions can exist at once. Arbitrary key-value metadata can be added for extensibility.

Groups are the glue binding clients and secrets. Clients are assigned membership into groups (e.g. for the service-on-host, service, etc.). Secrets are also assigned to groups. If a client and secret share any group, the client will have access to the secret. Groups are not hierarchical and cannot be in other groups.

Users are authenticated Keywhiz administrators. Authentication is customizable with LDAP and bcrypt-hashed passwords currently supported. After initial authentication subsequent requests require an encrypted cookie from Keywhiz Server.

Setup

Source code for the Server and CLI is available in square/keywhiz. Keysync source is in square/keysync. To checkout the Keywhiz source code:

$ git clone https://github.com/square/keywhiz.git && cd keywhiz

Starting the Server

Keywhiz stores data in MySQL and requires version 5.7 or higher.

From the base of the keywhiz repository, build the server:

$ mvn package -am -pl server

Run any migrations:

$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar preview-migrate server/src/main/resources/keywhiz-development.yaml $ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar migrate server/src/main/resources/keywhiz-development.yaml

Add an administrative user:

$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar add-user server/src/main/resources/keywhiz-development.yaml

Start the server:

$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar server server/src/main/resources/keywhiz-development.yaml

Building the CLI

From the base of the keywhiz repository:

$ mvn package -am -pl cli

Run the CLI and get a usage statement:

$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar

In development, you can use --devTrustStore , e.g.:

$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar --devTrustStore --user keywhizAdmin list groups

You may want to alias this command for convenience:

$ alias keywhiz.cli="/path/to/keywhiz-cli-*-SNAPSHOT-shaded.jar"

Setting-up Keysync

Refer to the Keysync README.

Examples

More examples are available in the Wiki, but here are a few.

In development, you can use --devTrustStore and these examples assume that you want to use the default keywhizAdmin user.

In production, you will want to create your own users and make sure that your certificates are properly signed and trusted.

Adding a Secret

Using Keywhiz CLI

$ keywhiz.cli --devTrustStore --user keywhizAdmin login $ keywhiz.cli add secret --name mySecretName < mySecretFile

Using Keywhiz automation API

The automation API requires a client certificate and automationAllowed=true in the clients DB table. For development purpose, you can use the pre-generated client.p12 keystore:

$ cat request.json { "name":"example.keytab", "description":"example kerberos keytab", "content":"a2V5dGFiIGNvbnRlbnQ=", "metadata":{"owner":"root","group":"root","mode":"0400"} } $ curl --cert ./server/src/test/resources/clients/client.p12:ponies -H "Content-Type:application/json" -d @request.json https://localhost:4444/automation/secrets/

Assigning a Secret to a Group

Using Keywhiz CLI

$ keywhiz.cli --devTrustStore --user keywhizAdmin assign secret --name example.keytab --group kerberos

Using Keywhiz automation API

$ curl --cert ./server/src/test/resources/clients/client.p12:ponies -X PUT "https://localhost:4444/automation/secrets/$SECRET_ID/groups/$GROUP_ID"

Download

↓ Latest JARs

Source code for Keywhiz and this website is available on GitHub.

Contributing

If you would like to contribute code to Keywhiz you can do so through GitHub by forking the repository and sending a pull request.

When submitting code, please make every effort to follow existing conventions and style in order to keep the code as readable as possible. Please also make sure your code compiles and passes tests by running mvn clean verify .

Before your code can be accepted into the project you must also sign the Individual Contributor License Agreement. We use cla-assistant.io and you will be prompted to sign once a pull request is opened.

License

Copyright 2015 Square, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.