June 2020 update: The second edition of these tutorials is now available! Details here.

At untapt, all of our models involve Natural Language Processing (NLP) in one way or another. Our algorithms consider the natural, written language of our users’ work experience and, based on real-world decisions that hiring managers have made, we can assign a probability that any given job applicant will be invited to interview for a given job opportunity.

A still from the intro to the “Deep Learning for NLP” video tutorials

With the breadth and nuance of natural language that job-seekers provide, these are computationally complex problems. We have found deep learning approaches to be uniquely well-suited to solving them. Deep learning algorithms:

trivially include millions of model parameters that are free to interact non-linearly; can incorporate learning units designed specifically for use with sequential data, like natural language or financial time series data; and, are typically far more efficient in production environments than traditional machine learning approaches for NLP.

To share my love of deep learning for NLP, I have created five hours of video tutorial content paired with hands-on Jupyter notebooks. Following on from my acclaimed Deep Learning with TensorFlow LiveLessons, which introduced the fundamentals of artificial neural networks, my Deep Learning for Natural Language Processing LiveLessons similarly embrace interactivity and intuition, enabling you to rapidly develop a specialization in state-of-the-art NLP.

A still from Lesson 3.2, where we calculate the area under the Receiver Operator Characteristic curve

These tutorials are for you if you’d like to learn how to:

preprocess natural language data for use in machine learning applications;

transform natural language into numerical representations (with word2vec);

make predictions with deep learning models trained on natural language;

apply advanced NLP approaches with Keras, the high-level TensorFlow API; or

improve deep learning model performance by tuning hyperparameters.

Below is a summary of the topics covered over the course of my five Deep Learning for NLP lessons (full breakdown detailed in my GitHub repository):

Lesson One: Introduction to Deep Learning for Natural Language Processing

high-level overview of deep learning as it pertains to NLP specifically

how to train deep learning models on an Nvidia GPU if you fancy quicker model-training times

summarize the key concepts introduced in my Deep Learning with TensorFlow LiveLessons, which serve as a foundation for the material introduced in these NLP-focused LiveLessons

Lesson Two: Word Vectors

leverage demos to enable an intuitive understanding of vector-space embeddings of words, i.e., nuanced, quantitative representations of word meaning

discover resources for pre-trained word vectors as well as natural language data sets

create your own vector-space embeddings with word2vec, including bespoke, interactive visualizations of the resulting word vectors

Word vectors enable meaning to be represented quantitatively, as illustrated here by Jon Krohn. The more similar the meaning of two words, the closer their representation in space, creating, e.g., a cluster of software languages, which here contains functional programming and object-oriented programming subclusters. Another feature of word vectors is that their distance and direction in space maps to meaning: In this cartoon, the green vectors represent gender while the brown vectors represent country-capital relationships.

Lesson Three: Modelling Natural Language Data

best practices for preprocessing natural language data

calculating the ROC curve to evaluate the performance of classification models

pair vector-space embedding with the fundamentals of deep learning introduced in my Deep Learning with TensorFlow LiveLessons to build dense and convolutional neural networks for classifying documents by their sentiment

Lesson 4.3 introduces Long Short-Term Memory Units (LSTMs)

Lesson Four: Recurrent Neural Networks

provide an intuitive understanding of Recurrent Neural Networks (RNNs), which permit backpropagation through time over sequential data, such as natural language and financial time series data

develop familiarity with the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) varieties of RNNs which provide markedly more productive modelling of sequential data

straightforwardly build LSTM and GRU deep learning architectures through the Keras high-level API

Lesson Five: Advanced Models