Starting off with a sample

api

storage

Getting down to action

The app-1 replica set will represent the api role that contains the vshard-router role. There will be just one instance.

replica set will represent the role that contains the role. There will be just one instance. The storage-1 replica set will represent the storage role (including the vshard-storage role) — here we will add two instances from different machines.

$ git clone https://github.com/dokshina/deploy-tarantool-cartridge-app.git $ cd deploy-tarantool-cartridge-app && git checkout 1.0.0

$ vagrant up

$ ansible-galaxy install tarantool.cartridge,1.0.1

$ ansible-playbook -i hosts.yml playbook.yml

Getting deeper into details

playbook.yml

--- - name: Deploy my Tarantool Cartridge app hosts: all become: true become_user: root tasks: - name: Import Tarantool Cartridge role import_role: name: tarantool.cartridge

tarantool.cartridge

--- all: vars: # common cluster variables cartridge_app_name: getting-started-app cartridge_package_path: ./getting-started-app-1.0.0-0.rpm # path to package cartridge_cluster_cookie: app-default-cookie # cluster cookie # common ssh options ansible_ssh_private_key_file: ~/.vagrant.d/insecure_private_key ansible_ssh_common_args: '-o IdentitiesOnly=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' # INSTANCES hosts: storage-1: config: advertise_uri: '172.19.0.2:3301' http_port: 8181 app-1: config: advertise_uri: '172.19.0.3:3301' http_port: 8182 storage-1-replica: config: advertise_uri: '172.19.0.3:3302' http_port: 8183 children: # GROUP INSTANCES BY MACHINES host1: vars: # first machine connection options ansible_host: 172.19.0.2 ansible_user: vagrant hosts: # instances to be started on the first machine storage-1: host2: vars: # second machine connection options ansible_host: 172.19.0.3 ansible_user: vagrant hosts: # instances to be started on the second machine app-1: storage-1-replica: # GROUP INSTANCES BY REPLICA SETS replicaset_app_1: vars: # replica set configuration replicaset_alias: app-1 failover_priority: - app-1 # leader roles: - 'api' hosts: # replica set instances app-1: replicaset_storage_1: vars: # replica set configuration replicaset_alias: storage-1 weight: 3 failover_priority: - storage-1 # leader - storage-1-replica roles: - 'storage' hosts: # replica set instances storage-1: storage-1-replica:

hosts.updated.yml

Managing the instances

ansible_host

ansible_user

hosts

storage-1

all: vars: ... # INSTANCES hosts: storage-1: config: advertise_uri: '172.19.0.2:3301' http_port: 8181 ...

config

advertise URI

HTTP port

app-1

storage-1-replica

host1

host2

vars

ansible_host

ansible_user

hosts

all: vars: ... hosts: ... children: # GROUP INSTANCES BY MACHINES host1: vars: # first machine connection options ansible_host: 172.19.0.2 ansible_user: vagrant hosts: # instances to be started on the first machine storage-1: host2: vars: # second machine connection options ansible_host: 172.19.0.3 ansible_user: vagrant hosts: # instances to be started on the second machine app-1: storage-1-replica:

hosts.yml

storage-2-replica

storage-2

all: vars: ... # INSTANCES hosts: ... storage-2: # <== config: advertise_uri: '172.19.0.3:3303' http_port: 8184 storage-2-replica: # <== config: advertise_uri: '172.19.0.2:3302' http_port: 8185 children: # GROUP INSTANCES BY MACHINES host1: vars: ... hosts: # instances to be started on the first machine storage-1: storage-2-replica: # <== host2: vars: ... hosts: # instances to be started on the second machine app-1: storage-1-replica: storage-2: # <== ...

$ ansible-playbook -i hosts.yml \ --limit storage-2,storage-2-replica \ playbook.yml

--limit

Managing the topology

storage-2

replicaset_storage_2

replicaset_storage_1

hosts

--- all: vars: ... hosts: ... children: ... # GROUP INSTANCES BY REPLICA SETS ... replicaset_storage_2: # <== vars: # replicaset configuration replicaset_alias: storage-2 weight: 2 failover_priority: - storage-2 - storage-2-replica roles: - 'storage' hosts: # replicaset instances storage-2: storage-2-replica:

$ ansible-playbook -i hosts.yml \ --limit replicaset_storage_2 \ --tags cartridge-replicasets \ playbook.yml

--limit

tags

cartridge-instances : instance management (configuration, membership);

: instance management (configuration, membership); cartridge-replicasets : topology management (replica set management and permanent removal (expel) of instances from the cluster);

: topology management (replica set management and permanent removal (expel) of instances from the cluster); cartridge-config : control of other cluster parameters (vshard bootstrapping, automatic failover, authorization parameters, and application configuration).

cartridge-replicasets

memtx_memory

vagrant halt

What's inside?

Installing the package and starting the instances

systemd

$ systemctl start myapp@storage-1

storage-1

myapp

/etc/tarantool/conf.d/

journald

/etc/systemd/systemd/myapp@.sevice

Configuring the cluster topology

systemd

The first option is to open the Web UI and click on the buttons. It is quite suitable for a one-time start of several instances.

The second option is to use GraphQL API. Here you can already automate something, for example, write a script in Python.

The third option is for the courageous: go to the server, connect to one of the instances with the help of tarantoolctl connect and perform all the necessary actions with the cartridge Lua module.

Results

group_vars

host_vars

We have already presented Tarantool Cartridge that allows you to develop and pack distributed applications. Now let's learn how to deploy and control these applications. No panic, it's all under control! We have brought together all the best practices of working with Tarantool Cartridge and wrote an Ansible role , which will deploy the package to servers, start and join instances into replica sets, configure authorization, bootstrap vshard, enable automatic failover and patch cluster configuration.Interesting, huh? Dive in, check details under the cut.Let us walk you through only some of the role's functions. You can always find a full description of all its features and input parameters in the documentation . However, trying once is better than seeing it a hundred times, so let us deploy a small application.Tarantool Cartridge has a tutorial for creating a small Cartridge application that stores information about bank customers and their accounts, as well as provides an API for data management via HTTP. For this purpose, the application describes two possible roles that can be assigned to the instances:andCartridge itself does not say anything about how to start processes — it only provides an opportunity to configure the running instances. So, the rest of it is up to the user: distributing configuration files, running services, and configuring topology. But we're not going to do all of that — Ansible will do it for us.First, let us deploy our application onto two virtual machines and set up a simple topology:To run the sample, we will need Vagrant and Ansible (version 2.8 or higher).The role itself is stored in Ansible Galaxy — a repository that allows you to share your work and use the ready-made roles.Now clone the sample repository:Then deploy the virtual machines:After that, install the Tarantool Cartridge Ansible role:And start the installed role:Now wait until the playbook process is finished, go to http://localhost:8181/admin/cluster/dashboard and enjoy the results:You can upload the data now. Awesome, isn't it?Now let us figure out how to work with it, and we may as well add another replica set to the topology.So, what happened?We got two virtual machines up and running and launched the Ansible playbook that configured our cluster. Now let's look inside thefile:Nothing interesting happens here; let's launch the Ansible role calledThe most important things (namely, the cluster configuration) is in theinventory file:All we need to do is learn how to manage instances and replica sets by modifying this file. Later on, we will add new sections to it. In order to avoid confusion while adding the sections, look at the final version of this file, or, which is located in the sample repository.In Ansible terms, each instance is a host (not to be confused with a physical server), i.e. the infrastructure node that Ansible will manage. For each host, we can specify connection parameters (such asand) and instance configuration. The instance description is in thesection.Let's look into the configuration of theinstance:In thevariable, we specified the instance parameters:andBelow are the parameters of theandinstances.We should provide Ansible with connection parameters for each instance. It seems reasonable to group the instances by virtual machines. For this purpose, the instances are grouped together underand, and each group in thesection contains theandparameter values for a single virtual machine. And thesection contains hosts (or instances) included in this group:Let's start editing. Now we add two more instances:on the first virtual machine andon the second one:Start the Ansible playbook:Note theoption. Since each cluster instance is a host in terms of Ansible, we can explicitly specify which instances should be configured when running the playbook.So we go back to the web UI at http://localhost:8181/admin/cluster/dashboard and look at our new instances:Next, let's master topology management.Let us group our new instances into thereplica set, add a new group of, and describe the replica set parameters in the variables as we did for. In thesection, we specify which instances should be included in this group (i.e. our replica set):Then we run the playbook again:This time we pass the name of the group corresponding to our replica set in theparameter.Let's look at theoption.Our role successively executes various tasks marked with the following tags:We can explicitly specify what part of the work we want to be done — and the role will skip the rest of the tasks. In this case, we only want to work with topology, so we specifyLet us evaluate the result of our efforts. Find the new replica set at http://localhost:8181/admin/cluster/dashboard Yay!Try changing the configuration of the instances and replica sets and see how the topology of the cluster changes. You can try different use cases, such as rolling update orincrease. The role would try to do this without restarting the instance to reduce the possible downtime of your application.Don't forget to runto stop the virtual machines when you're done with them.Here I will tell you more about what happened under the hood of the Ansible role during our tests.Let's consider the steps of deploying a Cartridge application.The first thing to do is to deliver the package to the server and install it. Now the role can work with RPM-packages and DEB-packages.Next, we launch the instances. It is very simple: every instance is a separateservice. For example:This command launches theinstance of theapplication. The running instance looks for its configuration in. You can view the instance logs usingThe Unit filefor the systemd service is delivered with the package.Ansible has built-in modules for installing packages and managing systemd services, so we did not invent anything new here.The most exciting things happen here. I am sure you would agree that it is strange to bother with a special Ansible role for installing packages and runningservices.You can configure the cluster manually:The main task of our invention is to do this most difficult part of the work for you.Ansible allows you to write your own module and use it in your role. Our role uses these modules to manage the various cluster components.How does it work? You describe the desired state of the cluster in a declarative configuration, and the role gives each module its own configuration section as input. The module receives the current state of the cluster and compares it with the input. Then the code for the necessary cluster state is launched using the socket of one of the instances.Today we have shown you how to deploy your Tarantool Cartridge application and configure a simple topology. To do this, we used Ansible, a powerful tool that is easy to use and allows you to configure multiple infrastructure nodes at the same time (in our case, the cluster instances).Above we went over one of the many ways to describe the cluster configuration by means of Ansible. Once you feel that you are ready for more, learn the best practices for writing playbooks. You may find it easier to manage the topology withandVery soon, we will tell you how to remove (expel) instances from the topology permanently, bootstrap vshard, manage automatic failover, configure authorization, and patch cluster configuration. In the meantime, you can review the documentation yourself and try changing cluster settings.If something goes wrong, make sure to let us know about the problem. We will do our best to resolve any issue!