Do you remember the first day you saw a computer terminal? No matter whether it was a Linux or Windows, the confusion it probably caused was huge. All those commands to remember, parameters to pass… Just to list your files or check the disk usage.

“Who needs that knowledge if I can click all of that? ” might have been your first thought.

But afterwards it turned out that using CLI instead of GUI is way more efficient and—let’s admit it—it looks awesome. The main difference between GUI and CLI tools is that GUI belongs to “learn how to use” category, while CLI is more like “learn how to configure”.

Nowadays—in the cloud times—I don’t have to worry about storing data like photos, music or code every time I change my computer. There is, however, one thing that I need to set up over and over again – my machine’s configuration. Here’s where the dotfiles repositories come in handy.

What’s best in dotfiles is that they are not only your backup, honestly this concept seems to be closer to a shared configuration. With dotfiles repo you can keep the same settings for your computers at home, at the office, remote servers or even share it with teammates.

Sounds nice, right?

Let me present you with a short set of useful preferences which you can fork from my repository (at the moment of writing this article its version is 1.6.0) and use on your own computer. I’ve created it for Ubuntu and OSX. All of the installation scripts for those modules are going to back up your current configuration and use symlinks to connect your configuration with files included in my repository.

Z Shell Stack

First of all, let’s run the terminal! For Debian distributions the shortcut for that is alt + tab + t. If your rather use OSX press ctrl + space, type “terminal” (or iTerm if you use it) inside the prompt and press enter. Now you probably see your default bash shell like this one:

It’s just basic. Nothing less than a fully functional shell and nothing more than a basic tty. Let’s change that!

We are going to switch from bash to Z Shell which is able to manage bash scripts but also do way more interesting things. There are many community frameworks which provide ready-to-work powerful ZSH configurations. A good practice is to use one of them if you don’t want to spend much time on configuration, but those solutions are a bit like closed boxes – it’s hard to connect them with external plugins.

That’s why we are going to use ZPlug to integrate with the library of the most popular framework called Oh My Zsh and in the process – with many awesome open source tools from GitHub. Let’s start!

Note: before running this script on Ubuntu please install gawk. It’s ZPlug’s dependency.

bash -c "$(curl -fsSL https://raw.githubusercontent.com/mkjmdski/.dotfiles/master/install.sh)"

Looks better now, right? It’s thanks to spaceship prompt theme, which is my default one (you can change it by setting the ZSH_THEME variable in .zshrc).

You can see that it displays useful information like:

Your current branch

Docker version

Pythonenv

Git status

It also works well with other virtualenvs, VCS and even AWS Profiles. To make it work, my script has powerline fonts installed which is a very popular dependency of most ZSH themes. Let’s check a bit of the installed configuration now.

Configuration Structure

There are two main files: the first one is called .zshrc which is located in your $HOME directory, and is responsible for the Z Shell configuration. It’s sourced once a user logs in. If you want to share this file with all the other users of the machine, you should move it to /etc/zshrc location.

The second file’s name is .zplugs.zsh which is loaded by .zshrc and it stores the configuration for ZPlug plugins. If you want to learn more about sourcing files flows in ZSH, check this article on ZSH startup. In order to see files loaded by your ZSH, run:

zsh -o SOURCE_TRACE

Zshrc configuration

Zshrc is an rc file like bashrc or shellrc. It should contain all PATH definitions, custom aliases and exports for system variables.

ZSH contains a really useful system variable called commands. You can use it to check if a program is loaded to the system. I have used it to resolve custom bin paths to add:

# Add go binaries if [ -d "$GOPATH" ]; then export PATH="$GOPATH/bin:$PATH" elif [[ $commands[go] ]]; then export PATH="$(go env GOPATH)/bin:$PATH" fi # Add yarn global binaries if [[ $commands[yarn] ]]; then export PATH="$(yarn global bin):$PATH"; fi

Here you can see how to set your default language and the files editor.

function _get_editor { echo $(which Vim) || echo $(which vi) } export EDITOR="$(_get_editor)" export LANG=en_US.UTF-8 # Default language

ZPlug configuration

Now let’s see some snippets from the zplug itself. For instance, this code allows zplug to update itself. Zplug first downloads its own repository, and later the hook which enables auto updating is evoked.

zplug 'zplug/zplug', hook-build:'zplug --self-manage'

Basing on the OS type, this part of configuration installs the correct gopass (I’ll talk about the gopass itself later) binary from its GitHub releases (check it to see how the _gopass_release function works).

function _gopass_release { [ "$(uname)" = "Linux" ] && echo '*linux*amd64*tar.gz' || echo '*darwin*' } zplug "gopasspw/gopass", from:gh-r, as:command, use:"$(_gopass_release)"

Here you can see how to load the library from the oh-my-zsh framework together with its minimal configuration (not all of the functions are loaded).

HIST_STAMPS="mm/dd/yyyy" # variable used in oh-my-zsh/lib/history.zsh zplug "robbyrussell/oh-my-zsh", use:"lib/{clipboard,completion,directories,history,termsupport,key-bindings}.zsh"

The last snippet shows how to load the autocompletion oh-my-zsh plugins (but they also could be not connected with any framework) basing on programs you have installed on the current profile.

zplug "plugins/docker", from:oh-my-zsh, if:'[[ $commands[docker] ]]' zplug "plugins/docker-compose", from:oh-my-zsh, if:'[[ $commands[docker-compose] ]]'

Brews, zplugs and gems: tap the best soft

Now, when you know how to configure zplugs, I will show you how to keep your software dependencies inside the repository without losing time for checks on startup. It’s very important, especially if you spawn around one hundred shell sessions daily (like I do).

Brew

When you need to get a new executable, software package managers come in handy. Many distributions have their own ones (like apt for Ubuntu or yum for Fedora/CentOS), but for some time now we have an awesome cross-platform tool for that.

Let me present you brew together with its linux port, linuxbrew. If you’ve installed dotfiles from my script, one of them is ready to work on your computer already. Now, if you need a new program (for e.g. the silver searcher) just add one line to the Brewfile.

brew “the_silver_searcher"

Done! Now when you run brews_install from installers.zsh your local packages will be checked against Brewfile and the difference will be updated. If you want to do it on each startup, just set BREW_UPDATE variable to true in .zshrc.

if ! brew bundle check --verbose --file=${DOTFILES}/Brewfile; then _log_info "Install missing brew formulas? [y/N]: " # Prompt about installing plugins if read -q; then echo; brew bundle install --file=${DOTFILES}/Brewfile fi fi

ZPlugins

I’d like to present you with a list of curated plugins I have included with my ZPlug. It’s highly possible that you don’t need some of them or you’d like to include other ones. Well, that’s the best reason why dotfiles should be forked.

Most of my ZPlugins are kept as so called “commands” inside $ZPLUG_BIN directory. It means that after they are installed they don’t have to be loaded (unless they are going to be updated). If you want to do it, set $ZPLUG_UPDATE variable to true.

if [ ! -d ~/.zplug ]; then git clone --depth=1 https://github.com/zplug/zplug ~/.zplug; fi export ZPLUG_LOADFILE="$DOTFILES/zsh/.zplugs.zsh" source ~/.zplug/init.zsh zplug load if [ "$ZPLUG_UPDATE" = true ] ; then zplugs_install zplug update fi

Luckily in ZPlug, including or excluding the program is a matter of just a few lines. Configurations presented below correspond to sections in .zplug.zsh commented with the same titles.

Vimode for zsh

Do you like Vim? Well, feel amazed because you can type your shell commands in Vim-style using this config! You can use all three of Visual, Insert and Normal modes, and hjkl arrows to navigate through the history.

Zsh magic

I called this set of plugins magic, because indeed it’s a magical experience. It adds syntax highlighting to the commands you type, colors your manual pages, adds autosuggestions based on what you had typed before and allows you to search history with commands you have typed using only up/down arrows (or “j” “k” in normal vimode).

Better system navigation

There are two basic commands in *nix systems: ls and cd. Forget about them. Now with exa/colorls and autojump browsing your files is way easier. Exa colors your ‘ls’ output and gives you a tree functionality. Autojump remembers how you change your directories and later you just need to type j <directory name> and the program finds this directory automatically so you don’t need to go through the full file-tree again.

Parsing outputs

I assume you were trying to parse your outputs many times with grep, awk and other GNU tools. That’s fine, especially as this is the standard approach. When you are inside a container, you probably don’t have any other solutions. But on your own computer? Say hello to peco and jq. Peco allows you to perform an easy search on outputs without using complex regular expressions, while jq is a great tool for getting data out of JSONs.

Gopass

Last but not least, a tool which I have included in my setup is gopass. This program, written in golang, is a modern version of good old pass. It allows teams to share secrets kept in .git repositories encrypted with GPG. That’s great when you need to maintain sensitive data, but you don’t necessarily want to set up a separate vault for them.

Gems

Some soft can’t be installed from brew or zplug integration with github. I created function called gems_install which checks your local gems and install those uninstalled. If you want to run it on each startup, just set GEMS_UPDATE variable to true. What’s best – you can adjust this function for npm, composer or any other package manager of your choice.

Git: may the –force be with you!

Git is a great tool which nowadays is shipped with nearly each of the available operating systems. It’s ready to work instantly but this master of VCS could also acquire some additional, craftsman guru setup. Enter .dotfiles/git and run install.sh. Now, both global.gitconfig and global.gitignore are included in your local configuration.

Few tricks from the config

Let’s say that you need to clone a repository from Bitbucket, and you know the path, but you are too lazy to write the full URL neither find it in the browser.

[url "git@bitbucket.org:"] insteadOf = bb:

Now you can just type in your shell:

git clone bb:apptension/project.git

Or another example: it’s a good practice to always rebase during the pull, because you can avoid having artificial merge commits in your history. Tired of typing git pull –rebase all the time?

[pull] rebase = true

Done! Tired of pushing tags in a separate command?

Aliases

Git allows us to create aliases. It’s a really nice utility, if you know your most used git combinations. As a DevOps Engineer I have to init repositories with some infrastructure setup. After generating templates I just run git this. What is git this?

this = !git init && git add -A && git commit -m \"Initial commit.\"

Or let’s say it’s time to clean up your work before pushing commits to the remote.

undo = reset --soft HEAD^ amend = commit --amend --no-edit log-line = log --oneline --graph --decorate

Git integrate

Git smoothly integrates with external tools in zplug setup you can find a tool called icdiff. It’s my replacement for the standard git diff command, the output of which is pretty unreadable. By using git-difftool variables icdiff is able to analyze data in the same way that git does, but in more readable way.

renames = true tool = icdiff [difftool] prompt = false [difftool "icdiff"] cmd = icdiff --line-numbers $LOCAL $REMOTE

Let’s check the results of this integration!

We can do the same integration for mergetool. I will use PyCharm because I love VCS tools shipped by JetBrains.

[merge] tool = pycharm [mergetool "pycharm"] cmd = /usr/local/bin/charm merge "$LOCAL" "$REMOTE" "$BASE" "$MERGED"

Done!

How to exit Vim and come back

Vim experience

To make it clear – I’m not going to try convincing anybody that Vim is better than graphical IDEs. Both of them have their own advantages and disadvantages and probably that’s why JetBrains have a plugin for using Vim inside their tools, and why there are projects that add graphical interface to Vim.

But there are use cases when I prefer using Vim over VS Code and as a newbie I wanted to have a bit more friendly experience with this program. If you want to edit a Python file in plain Vim you will see something like this:

It ain’t user friendly at all. And probably that’s why many people don’t like Vim’s in the first place. But luckily the community around Vim is huge and those people have developed many useful plugins.

To manage the add-ons we obviously need some plugin manager. In this case it’s Vundle. My installation script will link .vimrc to the one in a repository and run vundle to install all plugins. After this process, the same Python file looks like this:

Syntax highlighting, error console, line numbers, you can even scroll it with your mouse! As a terminal editor it’s way more useful now, isn’t it?

So when actually do I want to use Vim? Mostly for ssh purposes. When you need to edit remote server configuration you are going to enter Vim or nano anyway. It’s just better and easier to do it with your local setup. All you need to do is execute a command like this one:

vim scp://remoteuser@server.tld//absolute/path/to/document

Vim configuration

Configuring this tool is actually a piece of cake. Most of plugin managers for Vim are compatible with most implementations of Vim and using them is really easy. You just need to find a repository on GitHub with the plugin you need and stick to this easy schema:

set nocompatible " be iMproved, required filetype off " required " set the runtime path to include Vundle and initialize set rtp+=~/.vim/bundle/Vundle.vim call vundle#begin() " alternatively, pass a path where Vundle should install plugins " call vundle#begin('~/some/path/here') " let Vundle manage Vundle, required Plugin 'VundleVim/Vundle.vim' Plugin 'bash-support.vim' " All of your Plugins must be added before the following line call vundle#end() " required filetype plugin indent on " required

Also remember that each command you can execute in Vim, like this for showing line numbers:

:set number

can be added to your .vimrc to get loaded on program start.

Your editor – keep it remote

So now, when we know how to set up a good Vim configuration, it is time to do the same thing for the notepad. Nowadays there are three most popular lightweight, hackable code editors for programmers: Visual Studio Code from Microsoft, Atom from Github and Sublime.

I have tried all of them and my personal choice is VS Code. It starts up fast, even if many extensions are loaded, is open source and has many useful plugins from the community. All settings are kept as a one JSON file. For example, this is how you turn on autosave (same like in heavy JetBrains software):

{ "files.autoSave": "afterDelay", "files.autoSaveDelay": 1000 }

If you want to link the settings and snippets directory (VS Code allows you to create predefined code blocks) of your vscode to the repository just enter the vscode directory in dotfiles and run install.sh. This will also prompt you about installing extensions listed in installed_vs_extensions. So now you probably think: ok, so you want me to add each extension to this file when I install one?

No!

This is the place when git hooks come to work. Git hooks are scripts evoked after specific git actions designed to help developers maintain the automatically generated part of the repository (or automatic actions). Those files are kept inside the .git/hooks folder so you can’t push them directly to the remote, but anyway you can set them explicitly by running install.sh inside .hooks/ directory.

I have designed three hooks:

Pre-commit: lists all installed extensions to the cache file. Post-commit: moves cache file’s content to the installed_vs_extensions file. Post-merge (run after pulls too): checks which of extensions listed in the file are not installed and prompts user if he wants to install them.

Graceful shell usage

Terminal multiplexing

Normally in basic terminal emulators you have one shell session per each tab. This is not efficient because you need to switch between tabs to control those sessions. Let’s change that!

For Linux users I recommend installing terminator and for Mac users iTerm 2. Terminal freaks could also use tmux. Now you are able to manage a few sessions in one tab, turning your terminal into a really powerful tool:

Shell tricks

Let me show you a few tricks I use when working with my setup daily:

Managing directories

cd /Users/me/Documents/devops/graylog #it goes to the directory /Users/me/Documents/devops/graylog #it goes to the directory in oh my zsh j graylog #autojump finds this directory if you’ve been there before - # previous directory 3 # three times previous directory … # two directories up …. # three directories up mkdir -p /usr/newbin/{groovy_bins,ruby_bins} #create two directories inside newbin

Extracting archives and files

tar -zxvf archive.tar.gz #it unpacks tarball in debian gunzip archive.tar.gz | tar xopf - #it unpacks tarball in osx x archive.tar.gz #it unpacks tarball on both x archive.zip #works with zip, rar and others too cat file # probably that’s how you get file content to copy it inside clipboard clipcopy file #not anymore # wanna paste? clippaste

Caching shell sessions

Let’s say you need to set up three identical servers. The first one you want to configure manually – to learn the environment, but the other two will be provisioned automatically. Use the script command!

script my.terminal.session echo $PWD date sudo apt install -y gawk

Now you can review all of your steps by simply running:

cat my.terminal.session #all session redirected to output less my.terminal.session #see the last steps of your session more my.terminal.session #see first steps of your session

Repeating commands and standard variables

Now when you have zsh magic it won’t be hard, because autosuggestions and search history can help you a lot. But still, there are a few standard shell tricks which can improve providing input into shell.

!! # last command !1345 # command number 1345 from your history !docker #last command which contains word docker $? # returns exit code of last command !$ # last word from last command. Useful when you want to enter file again with different program

Thanks for your attention! Remember about the last command.

leave +0330 # Reminds you that you have to leave in 3 hours and 30 minutes