5 Powerful PyTorch Functions Every Beginner Should Know

Torch your Data Science world alight with these Tensor related functions

“How many Marks did you get in your data science class?” “10 out of tensor” | Photo by Ales Nesetril on Unsplash

PyTorch proficiency is one of the most sought after skill when it comes to recruitment for data scientists. For those who don’t know, PyTorch is a Python library with a wide variety of functions and operations, mostly used for deep learning. One of the most basic yet important parts of PyTorch is the ability to create Tensors. A tensor is a number, vector, matrix, or any n-dimensional array.

Now the question might be, ‘why not use numpy arrays instead?’

For Deep Learning, we would need to compute the derivative of elements of the data. PyTorch provides the ability to compute derivatives automatically, which NumPy does not. This is called ‘Auto Grad.’ PyTorch also provides built in support for fast execution using GPU. This is essential when it comes to training models.

All Deep Learning projects using PyTorch start with creating a tensor. Let’s see a few MUST HAVE functions which are the backbone of any Deep Learning project.

• torch.tensor()
• torch.from_numpy()
• torch.unbind()
• torch.where()
• torch.trapz()

Before we begin, let’s install and import PyTorch

Function 1 — torch.tensor

Creates a new tensor. Arguments taken are :

1. Data: The actual data to be stored in the tensor.
2. dtype: Type of data. Note that type of all the elements of a tensor must be the same.
3. device: To tell if GPU or CPU should be used.
4. requires_grad: If you want the tensor to be differentiable (to compute gradient), set to True.

Returns a tensor object.

Example 1 (Working Example) :

This creates a tensor with shape (10,2)

Example 2 (Working Example) :

We can also create an empty tensor using the same function.

As we can see, an empty tensor with size (0) is created.

Example 3 (Error) :

A tensor with string values cannot be created.

Summary :

torch.tensor() forms the core of any PyTorch project, quite literally, as it forms tensors.

Function 2 — torch.from_numpy

For those who work with NumPy Arrays, this is a vital function. Using this, one can convert a NumPy Array into a Tensor. It is to be noted that as the tensor and the NumPy Array share the same memory, any changes made to the tensor would be applicable to the NumPy Array and vice versa. The arguments taken are :

1. A NumPy ndarray

Returns a tensor.

Example 1 (Working Example) :

A NumPy array has been converted into a PyTorch Tensor. We can verify that by checking the type of both a1 and t1 as shown above.

Example 2 (Working Example) :

We first created two NumPy Arrays which contained (height, weight) and (heart rate) of 3 students. We created a tuple of both the variables. Then using a for loop, we converted the elements of the tuple to a tensor one by one. In the end, we checked the type of the elements to verify if the function has worked.

Example 3 (Error) :

A NumPy array containing string elements cannot be converted to a tensor. The only supported types for a tensor are : float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.

Summary :

This function is extremely useful for people who are using NumPy Arrays for their projects. It shows the interoperability of different Python libraries for data science which makes the language a go to for the majority of data science enthusiasts.

Function 3 — tensor.unbind

This function removes a dimension from a tensor and returns a tuple of slices of the tensor without the removed dimension. Arguments it takes are :

1. A tensor
2. Dimension to be removed (0 by default)

Returns a tuple of slices.

Example 1(Working Example) :

The 0th dimension from (0,1) is removed thus we get 3 slices of the tensor in a tuple

Example 2 (Working Example) :

As seen above, now the data is sliced into data of 10 students for each day and stored in a tuple of tensors, that is, 7 different tensors are created, each corresponding to a day of the week. This sliced data can be used to further apply logic based on that particular day of the week.

Example 3(Error) :

We had created a tensor with 2 dimensions. We provided a value of 2 to the tensor.unbind() function. As 2 corresponds to the 3rd dimension, which doesn't exist in our example, the error 'Dimension out of range' is produced.

Summary:

This is a powerful PyTorch function that could be useful when you want to work on particular slices of the data along a dimension of the tensor.

Function 4 — torch.where

This is a really useful conditional function which depending on a condition, a returns a tensor of elements selected from either x or y tensor. Arguments required are :

1. A condition
2. x: Elements from this tensor are selected for indices where Condition is True
3. y: Elements from this tensor are selected for indices where Condition is False

Example 1(Working Example) :

Here we gave a condition where the elements from X which were positive, got selected. The elements which were not positive were replaced with elements from Y

Example 2 (Working Example):

Here we compared head to head scores and got a tensor containing scores of the winners.

Example 3(Working Example):

It is vital for dimensions and/or size of both the tensors x and y to match for comparison. Otherwise, we get the above error.

Summary :

Conditional testing is very important for any data analysis. We might need a particular type of data from two sets of data based on a certain criteria. torch.where() is a useful function for such scenarios.

Function 5 — torch.trapz

This function estimates the definite integral of y with respect to x along the given dimension, based on 'Trapezoidal rule'.

The arguments required are :

1. y : A tensor containing values of the function to integrate. (blue line in the illustration below)
2. x : The points at which the function y is sampled. (x axis in the illustration below)
3. Dimension for integration

Returns a tensor of the same shape as the input (except the dimension removed) where each elements represents the estimated integral along the given dimension.

Example 1 (Working Example) :

Here we have the estimated definite integration of y with respect to x.

Example 2 (Working Example) :

We can also provide a dx argument where sample points are spaced uniformly at a distance of dx.

Example 3 (Error) :

As we can see, the dimensions and size of both y and x should match, otherwise our code will produce the above error.

Summary:

Deep Learning requires a lot of calculus to know more about the models that are being used. torch.trapz() makes our job of finding estimated integral easy.

Conclusion

This concludes our look at 5 important PyTorch functions. From basic tensor creation to advanced and lesser known functions with specific usecases like torch.trapz, PyTorch provides many such functions which make job of a data science enthusiast easier. This was a beginner friendly introduction to PyTorch. There is much more. From here it would be a good idea to explore the documentation and create your own tensors. Play around and have some fun. As we get a tighter grip on the basics, we can move forward to Neural Networks and Deep Learning.

Check out the ongoing course regarding Deep Learning by jovian.ai, Zero To Gans.

Also check out my beginner friendly data analysis of Kickstarter Projects using Pandas, Matplotlib and Seaborn.

Check out similar 5 powerful functions for beginners provided by NumPy here.

Reference Links

• Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
• Deep Learning with PyTorch: Zero to GANs by Jovian.ai: https://jovian.ai/learn/deep-learning-with-pytorch-zero-to-gans