Linux’izing your Windows PC into a dev machine – Part 2

43 shares







(Hey, for Part 1 have a look here: https://cepa.io/2018/02/10/linuxizing-your-windows-pc-part1/)

Last week I presented how to setup basic Linux features on Windows through the Bash On Windows aka Windows Subsystem for Linux (WSL). You should be now able to run variety of Linux apps both command line and with GUI. Also, presented how to install Docker and Vagrant which are popular tools to isolate your host environment from development playground.

Let’s install essential tools for variety of modern software development technologies and get prepared for some hardcore coding battles!

Reading feedback from comments on various websites really helped to select what tools should I try to install and run in WSL. I’m not actively using all of the languages and technologies tested down below, so feel free to comment if something isn’t right or perhaps you have some suggestions.

The Part 2 of Linux’izing Windows will be focused mostly on Web, DevOps and fancy Machine Learning stuff. I have skipped the Desktop or Mobile development as it is already perfectly doable on native Windows via dedicated tools, languages and technologies. Ok, maybe with exception or Linux desktop development, but that should just work with the X Server regardless of using Qt, GTK, or any other toolkit. OSX or iOS? Well for that you’ll need a Mac anyways.

The following is the table of contents from Part 1 and Part 2, you can jump straight to your weapon of choice.

Part I

Part II

Sharing environment between Windows and WSL

Probably should have started by desrcibing this last week, anyway, here it goes.

So, you install the WSL on your Windows PC and then what?

Where is WSL installed?

Your Linux filesystem will be created in your user’s AppData directory, preciesly in C:\Users\your.username\AppData\Local\lxss. It is a hidden folder, inside you’ll find root, home and other standard Unix directories, you should avoid manual changes there from outside of WSL.

How to share files between Windows and WSL?

Guess, there are plenty of ways, I’ll show you how I do it.

By default WSL has access to your Windows drive, it is mounted in /mnt/c, for example your Windows home directory is C:\Users\your.username and in WSL this directory is available at /mnt/c/Users/your.username.

I keep all my projects in the projects folder in my Windows home directory C:\Users\cepa\projects. When you start bash you’ll get into your WSL home directory which is located in WSL filesystem, however you can create a symlink to your projects folder.

cd ~ ln -s /mnt/c/Users/your.username/projects projects cd projects

Accessing other disks and network locations?

Normally on Linux you would directly mount a filesystem from a disk using the mount command and a Windows network location using Samba. In WSL it is a bit different though.

WSL filesystem uses DrvFs which seems to be a Microsoft’s invention to share filesystem between Windows host and WSL layer, and apparently, you can use DrvFs to mount other Windows drives and network locations.

You can have a look here for more details:

https://blogs.msdn.microsoft.com/wsl/2017/04/18/file-system-improvements-to-the-windows-subsystem-for-linux/

Quick how to. Let’s say you have a drive D: in your Windows PC and to mount it run: sudo mkdir /mnt/d mount -t drvfs D: /mnt/d -o metadata,uid=1000,gid=1000,umask=22,fmask=111 Now you can cd /mnt/d and access files. That works also with network locations mounted in Windows as shared drives.

For network locations you have two options, first mentioned above, just mount it as a drive, second is mounting directly in WSL: sudo mkdir /mnt/share sudo mount -t drvfs

The df -h works too:



Getting basic Linux utilities and services

Lets get ready for upcoming coding battles!

First, you can install bunch of standard Linux tools which are often required while installing other software packages or are just handy.

sudo apt-get install -y \ apt-transport-https \ lsb-release \ ca-certificates \ build-essential \ curl wget unzip zip tmux git

It will get you the GCC compiler, Git and bunch of other utilities. There are lots of other tools available and for most you can follow Ubuntu how to’s to install them.

Accessing your Windows PC over SSH

This one is pretty cool 😉

With WSL you not only can use SSH to access remote Linux servers but you can actually run OpenSSH server in your WSL to access your Windows PC from outside! Just like any regular Linux box.

So, OpenSSH server on WSL:

Reinstall OpenSSH in WSL sudo dpkg-reconfigure openssh-server Edit SSH configuration, sudo nano /etc/ssh/sshd_config, and change the following parameters: Port 2222 # ... UsePrivilegeSeparation no # ... PasswordAuthentication yes Please be aware that it allows to use password authentication which is not the best idea, if you need SSH server please read about SSH hardening and password-less authentication. Open Windows firwall for a custom port (2222)

In Windows 10 you might have an SSH Server Broker Services running which occupy port 22, thus the change to port 2222. You’ll need to open Windows 10 firewall to allow incoming connections to your SSH server.

– Type WF.msc in the Start menu

– In the Windows Firewall with Advanced Security click on Inbound Rules

– Add a new rule for TCP 2222 and allow the connection Restart SSH serwer in WSL sudo service ssh --full-restart SSH into your Windows PC over from a remote host ssh your.wsl.username@your.windows.pc.ip -p 2222

Done, should just work.

LAMP stack for PHP development

PHP is an amazingly popular technology for creating various web projects. Many developers share the love-hate relationship with it due to its low entry and loads of crap code around and projects that are total mess from code perspective. WordPress anyone?

Before WSL times, you had basically two options, either use Linux and develop in PHP on a production like environment or struggle with WAMP bundle which is an acronym for Windows Apache MySQL PHP.

With WSL you can have a regular LAMP – Linux Apache MySQL PHP – setup on your Windows PC that is very similar to production Linux based environment where your PHP application is most likely to be hosted.

Apache HTTP Server with PHP 7

This is going to be a pretty standard setup, you can of course install Nginx+PHP FPM or other variations, up to you.

Install Linux packages for Apache and PHP sudo apt-get install \ apache2 \ libapache2-mod-php7.0 \ php7.0 \ php7.0-cli \ php7.0-common \ php7.0-curl \ php7.0-dev \ php7.0-gd \ php7.0-json \ php7.0-mysql \ php7.0-opcache \ php7.0-xml \ php7.0-bz2 \ php7.0-intl \ php7.0-mbstring \ php7.0-mcrypt \ php7.0-zip Enable additional Apache modules like mod_rewrite sudo a2enmod rewrite Restart Apache service sudo service apache2 restart Create sample PHP file, run: sudo nano /var/www/html/phpinfo.php and write: <?php phpinfo(); Ctrl+X Open http://localhost/phpinfo.php, you should get the PHP Info page.



MySQL Server

On Windows PC you can either have a MySQL server installed as a regular Windows service or inside WSL, here goes the WSL way:

Install Linux MySQL packages: sudo apt-get install mysql-server mysql-client Start MySQL service: sudo service mysql start Open MySQL command line client, run sudo mysql it should display a MySQL prompt, can type: show databases; which should print something like:

You can use MySQL Workbench and manage your WSL’s MySQL using it, use localhost:3306 for the connection.

Composer

Composer is a de-facto standard tool for PHP dependency management. You can easily install it in WSL, for more details visit https://getcomposer.org/download/ here’s the short version:

Install Composer php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" php composer-setup.phpphp -r "unlink('composer-setup.php');" Make it global sudo mv composer.phar /usr/local/bin/composer chmod +x /usr/local/bin/composer Run it



Symfony Project

So, far so good however, a simple phpinfo.php isn’t really a PHP project.

Let’s try something heavier, a lot heavier…

Symfony is probably the most popular enterprise-grade PHP framework of today. Typical boilerplate project is made of around 2800 files and weights around 8MB, that’s quite a lot of files to process, which makes it… a bit slow 😉

Details are available here https://symfony.com/doc/current/best_practices/creating-the-project.html but for sake of this blog post let’s do it quick, you can create a fresh Symfony project using Composer mentioned above:

Create a new Symfony project composer create-project symfony/skeleton sftest It will take a while to finish… Run the built-in PHP test server cd sftest php -S 127.0.0.1:8000 -t public Open http://localhost:8000/ in a browser on Windows, you should get Symfony project page.



MEAN stack for Node.js development

MEAN stands for MongoDB Express.js Angular Node.js.

JavaScript and Node.js are the backbone of modern frontend development. With WSL you can easily install Node.js ecosystem and use just like on Linux or MacOSX.

Node.js v9 and NPM

For details you can have a look here https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

Install Node.js on WSL curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash - sudo apt-get install nodejs Avoid permission errors (EPERM) with NPM by setting NPM to use a folder owned by your user to install Node.js components and binaries mkdir ~/npm sudo npm config set prefix ~/npm sudo chown -R $USER:$(id -qn $USER) ~/.config Add new NPM location to your WSL’s PATH variable echo 'export PATH="~/npm/bin:$PATH"' >> ~/.bashrc source ~/.bashrc Update NPM packages npm install -g npm Install common Node.js components (optional)

Once Node.js and NPM are installed you can install bunch of common Node.js utilities npm install -g typescript bower grunt gulp less sass yarn webpack

So, it looks good but again, let’s try something heavier.

Angular in WSL

NPM is there so you can try with a new boilerplate Angular project.

Install Angular CLI npm install -g @angular/cli Well… it will download half of Internet of dependencies so be patient… Create a new Angular project ng new ngtest Build and run Angular in Node.js server cd ngtest ng serve --host 0.0.0.0 --disable-host-check Open http://localhost:4200/ on Windows, you should get a blank Angular 4 project page.



React in WSL

There seem to be a religious war between Angular and React devotees so I shouldn’t forget about the other pillar of frontend stack, again, with NPM it’s easy.

Install React’s creator npm install -g create-react-app Wait… again… the other half of Internet of dependencies… Create a new React project create-react-app reacttest Build and run React in Node.js server cd reacttest npm start Open http://localhost:3000/, interestingly React in WSL has triggered my Windows default browser to automagically open that link for me.



MongoDB in WSL

Well, I admit, this one was a bit of pain…

Generally you can install and use MongoDB in WSL, there are some issues with the way the MongoDB service is being run in WSL/Linux.

The following is based on the officiall how to https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

Instal Mongo Community Edition 3.6 sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5 echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list sudo apt-get update sudo apt-get install mongodb-org Unfortunately, Mongo won’t start as a service, seems to be an issue with Systemd/Dbus, but you can run is the other way. You can start mongod as mongodb user using su: sudo su - mongodb -s /bin/bash -c "/usr/bin/mongod --config /etc/mongod.conf" Connect to Mongo server using Mongo client:

Optionally you can use external clients like Mongo Booster and connect to Mongo server from a Windows app.



Python, Django and Machine Learning

Python is a fabulous language. I hardly can think of any project that can’t be built with it, anything from web applications, through cloud, image processing to machine learning. You can do it all with Python and the language itself is super easy to learn.

Python has its versions available for Windows, however the power of this language lies in the thousands of libraries created by community. Many of these libraries are either dependant of native libraries available on Linux or require direct device access for some features and that seem to be an issue in many cases while using a native Python for Windows.

It seems WSL has solved some of these issues. Although, you won’t get direct access to devices like GPU for say Machine Learning – at least I’m not aware you can – however, with WSL you can use libraries that are difficult to install natively on Windows, including these for mentioned Machine Learning.

Python in WSL and Virtualenv

First, let’s install Python in WSL and additionally virtualenv to manage project specific environments. This way you can have multiple version of Python in your projects, like Python 2.7 and Python 3.

sudo apt-get install python python-dev python-setuptools sudo easy_install pip sudo pip install virtualenv

Now run Python to confirm it works:

python --version

Good, now can build some Python projects.

Django in WSL

As a starter, can try with a fresh Django project.

Create a new virtualenv with Python 3 for a Django project virtualenv -p python3 djangotest Activate virtualenv cd djangotest source ./bin/activate Install Django pip install Django Create new Django project django-admin startproject myapp Start Django cd myapp python manage.py runserver Open http://localhost:8000/, should get the default Django app page.



Python Machine Learning in WSL

I’m neither Data Scientist nor Machine Learning expert so this section is based on information found in these two articles:

https://github.com/dzorlu/GADS/wiki/Guide-to-installing-machine-learning-libraries-in-python

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/

Create and activate a new virtualenv virtualenv pyml cd pyml source ./bin/activate Install NumPy pip install numpy Install SciPy pip install scipy Install Matplotlib sudo apt-get install libfreetype6-dev pip install matplotlib Install Patsy pip install patsy Install StatsModels pip install statsmodels Install ggplot pip install ggplot Install Sciki-learn pip install scikit-learn Install Theano pip install theano Install Tensorflow pip install tensorflow Install Keras pip install keras

Seems to be all right.

Could have installed them in a single pip instal … command, but wanted to make sure each or these works and catch potential issues.

Java and JVM ecosystem

When it comes to Java and JVM I would suggest installing it both direclty on your Windows and separately in WSL. Why? Because you’ll need Java for example to run IDEs like Eclipse or NetBeans and you could have a different version of Java in WSL.

First, I installed Oracle JDK 8 for Windows on my Windows host to run various Windows programs.

Second, I installed Oracle JDK 9 for Linux in my WSL. You can follow this guide https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

Short version:

Add Webupd8 repo sudo add-apt-repository ppa:webupd8team/java sudo apt-get update Install Oracle Java 9 sudo apt-get install oracle-java9-installer Run Java in WSL to check version java -version

Maven in WSL

Maven and Ant are common tools in Java development environment and installing it on WSL is straightforward.

sudo apt-get install maven mvn --version

Ok, so Java seems to be working fine, let’s try to build some real Java project.



Spring Boot in WSL

As an example you can use one of the Spring Boot samples which is a very popular Java framework. I’ve used the one from https://spring.io/guides#gs

Clone the project git clone https://github.com/spring-guides/gs-spring-boot Build it cd gs-spring-boot cd complete chmod +x mvnw ./mvnw spring-boot:run Open http://localhost:8080/, you should get a page saying “Greetings from Spring Boot!”



SDKMAN! in WSL

To install other components for JVM ecosystem you can use SDKMAN!

Installation is easy, follow the official documentation at http://sdkman.io/install.html

Install SDKMAN! curl -s "https://get.sdkman.io" | bash Add SDKMAN to the PATH source "$HOME/.sdkman/bin/sdkman-init.sh" Run and check version sdk version

Scala in WSL



To get Scala in WSL simply run SDKMAN.

sdk install scala scala -version

All good!



Gradle in WSL

Can try installing Gradle in WSL using SDKMAN.

sdk install gradle 4.5.1 gradle -v

All good!

Kotlin in WSL

Let’s try with Kotlin, again with SDKMAN.

sdk install kotlin kotlin -version

All good!

DevOps tools for Amazon AWS

With WSL you can easily run DevOps tools to manage your cloud infrastructure.

Azure tools should work out of the box in Windows, right Microsoft?

So, let’s get AWS tools up and running.

AWS CLI in WSL

If you work with Amazon AWS Cloud you’ll definitely need the AWS CLI which is the official command line client to manage your infrastructure in AWS cloud.

You can easily install it with Python Pip, see the Python section first and install Python. When done just run the following.

sudo pip install awscli aws configure

Follow the official documentation to configure it https://docs.aws.amazon.com/cli/latest/userguide/installing.html

You’ll need an AWS Account with Access Key and Secret Key which you can set up in IAM in the AWS Web Console.



Terraform and Packer in WSL

Ever heard of the Infrastructure as Code concept? Basically you use a meta language to describe how your infra should “look like” and that meta description is processed by some software that eventually will build up your servers, networking, storage, etc.

If you look for a new hobby in life you can use AWS CloudFormation and write ass long JSON objects to describe how you want your AWS infra to be.

But, it’s 2018 and there are better tools. Apparently, HashiCorp heard the moaning of DevOps hordes and inventend two tools Terraform and Packer to automate cloud operations. They aren’t perfect, but A LOT better than JSON and CloudFormation.

You have two options on Windows, you can use native Terraform and Packer builds for Windows or you can use Linux ones in WSL. It’s a blog post about WSL so I’ll use the Linux ones.

Install Terraform from https://www.terraform.io/downloads.html

Download the Linux 64bit version

Move it to the bin directory: mv terraform /usr/local/bin chmod +x terraform /usr/local/bin terraform -v Install Packer from https://www.packer.io/downloads.html

Download Linux 64bit version

Move it to the bin directory: mv packer /usr/local/bin chmod +x packer /usr/local/bin packer -v

Sample output:

Looks good!

Things that suck

Oi! Microsoft, you reading this? 😉

Can you guys please improve a couple of things?

IO is slow, very slow!

I haven’t done any real benchmark yet but generally using WSL feels sluggish, a lot slower than working on Linux. Subjectively the difference is 10x or so which is a serious issue.

The problem is especially while working with technologies like PHP or Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow.

Frankly, I’m working on two computers now. One laptop with WSL on it which is my primary Office/Mail/Remote work machine and a normal stationary workstation with multiple monitors and Linux on it. Whenever I have a heavier task to do I feel more comfortable working from my workstation. To be fair, it would be good to perform a benchmark of WSL on a real workstation rather than a low power laptop but still I bet the IO speed difference would be significant.

I haven’t done any real benchmark yet but generally using WSL feels sluggish, a lot slower than working on Linux. Subjectively the difference is 10x or so which is a serious issue. The problem is especially while working with technologies like PHP or Node.js where projects have many thousands of files and are processing them. On Linux that just flies, on WSL it’s often unbearably slow. Frankly, I’m working on two computers now. One laptop with WSL on it which is my primary Office/Mail/Remote work machine and a normal stationary workstation with multiple monitors and Linux on it. Whenever I have a heavier task to do I feel more comfortable working from my workstation. To be fair, it would be good to perform a benchmark of WSL on a real workstation rather than a low power laptop but still I bet the IO speed difference would be significant. Networking

Many sysadmin tools just don’t work. I guess that’s because the WSL kernel doesn’t have all the features and have to share networking with Windows. If you’re primary workload is development WSL is just all right, but if you mostly do network operations then you’d be way better on Linux or Linux VM.

Many sysadmin tools just don’t work. I guess that’s because the WSL kernel doesn’t have all the features and have to share networking with Windows. If you’re primary workload is development WSL is just all right, but if you mostly do network operations then you’d be way better on Linux or Linux VM. Lack of Systemd/Dbus and system wide startup

See the MongoDB bit of this article. Linux packages usually contains the software and the startup scripts. Often, the startup scripts depend on Systemd which is not supported(?) in WSL and that causes issues with starting Linux services. That being said, be aware that you need to keep at least one Bash on Windows instance running to keep your services like say Apache, MySQL or SSH alive.

See the MongoDB bit of this article. Linux packages usually contains the software and the startup scripts. Often, the startup scripts depend on Systemd which is not supported(?) in WSL and that causes issues with starting Linux services. That being said, be aware that you need to keep at least one Bash on Windows instance running to keep your services like say Apache, MySQL or SSH alive. Filesystem permissions

Bash on Windows has its own filesystem embedded into Window, however it is not fully compatible with typical Linux filesystem. You might have issues while working with Node.js or some other toolkits that need setfacl to work properly for various users other than your own which owns the current bash process.

Bash on Windows has its own filesystem embedded into Window, however it is not fully compatible with typical Linux filesystem. You might have issues while working with Node.js or some other toolkits that need setfacl to work properly for various users other than your own which owns the current bash process. Mounting external filesystems

DrvFs works but can’t mount things like NFS or even SSHFS via Fuse. That would be nice to have.

DrvFs works but can’t mount things like NFS or even SSHFS via Fuse. That would be nice to have. Direct device access

When you go to /dev there are only some dummy devices, can’t access drives or other hardware. Not sure if that would be possible to implement though.

Hot or not?

Am I going to ditch Linux completely?

No, at least not yet.

WSL is a great improvement over working on either a raw Windows 10 or Windows with Cygwin however there are times that you need a real not a castrated Linux like WSL. I think the 80/20 rule applies to it as well. For a regular daily use you can perform most of your tasks within WSL, but whenever you need some more advanced networking, device access or performance having a real Linux workstation is just better.

Personally I think WSL shines by bringing long missing features to Windows, it allows you to get a flexible machine both for office and development workstation use. With release of the build 1709, Windows 10 has became a real alternative for Mac OS X as all-in-one environment for developers and perhaps in time if it gets event better it might be a serious alternative for Linux desktop workstations.

Time will tell.

Other links:

Discussion on Hacker News (Part 1) https://news.ycombinator.com/item?id=16351716

Discussion on Hacker News (Part 2) https://news.ycombinator.com/item?id=16453751

WSL Benchmark by Phoronix https://www.phoronix.com/scan.php?page=article&item=wsl-february-2018&num=1

Feel free to post feedback in comments, social media, email, etc and thanks in advance for sharing this article.