PyTorch Tensors

PyTorch Tensors are very similar to NumPy arrays with the addition that they can run on the GPU. This is important because it helps accelerate numerical computations, which can increase the speed of neural networks by 50 times or greater. In order to use PyTorch, you’ll need to head over to their website to install it. If you’re using Conda, you can install PyTorch by running this simple command:

In order to define a PyTorch tensor, start by importing the torch package. PyTorch allows you to define two types of tensors — a CPU and GPU tensor. For this tutorial, I’ll assume you’re running a CPU machine, but I’ll also show you how to define tensors in a GPU:

The default tensor type in PyTorch is a float tensor defined as torch.FloatTensor . As an example, you’ll create a tensor from a Python list:

If you’re using a GPU-enabled machine, you’ll define the tensor as shown below:

You can also perform mathematical computations such as addition and subtraction using PyTorch tensors:

You can also define matrices and perform matrix operations. Let’s see how you’d define a matrix and transpose it:

PyTorch Autograd

PyTorch uses a technique called automatic differentiation that numerically evaluates the derivative of a function. Automatic differentiation computes backward passes in neural networks. In training neural networks weights are randomly initialized to numbers that are near zero but not zero. A backward pass is the process by which these weights are adjusted from right to left, and a forward pass is the inverse (left to right).

torch.autograd is the library that supports automatic differentiation in PyTorch. The central class of this package is torch.Tensor . To track all operations on it, set .requires_grad as True . To compute all gradients, call .backward() . The gradient for this tensor will be accumulated in the .grad attribute.

If you want to detach a tensor from computation history, call the .detach() function. This will also prevent future computations on the tensor from being tracked. Another way to prevent history tracking is by wrapping your code with torch.no_grad():

The Tensor and Function classes are interconnected to build an acyclic graph that encodes a complete history of the computation. The .grad_fn attribute of the tensor references the Function that created the tensor. To compute derivatives, call .backward() on a Tensor . If the Tensor contains one element, you don’t have to specify any parameters for the backward() function. If the Tensor contains more than one element, specify a gradient that’s a tensor of matching shape.

As an example, you’ll create two tensors, one with requires_grad as True and the other as False . You’ll then use these two tensors to perform addition and sum operations. Thereafter, you’ll compute the gradient of one of the tensors.