The world’s leading publication for data science, AI, and ML professionals.

What is the Shell in Linux?

Explaining the purpose and basic functionality of the shell

Introduction

I have recently started a new job and one of the key things I found myself using more than before is the shell.

During university I was exposed to the shell a little bit when I was writing FORTRAN code (yeah, thats right, FORTRAN). However, I didn’t really understand what the shell was or the commands I was using were actually doing (I was more concerned of not failing the assignment!).

In this post I would like to shed some light on the purpose of the shell and show some of its basic functionality.

Note: This is going to be on the MacOS shell, therefore most commands will probably not work on a windows machine. However, the principles are all the same!

What Is The Shell?

Let’s begin with discussing what the shell actually is.

Nowadays, we interact with the computer using a Graphical User Interface (GUI) to access files, programs etc. However, the GUI has certain limitations and doesn’t allow you to use the full potential of the computer.

To take the full advantage of your computer, we need to use the text based interface, the shell. The shell is also known as a Common Line Interface (CLI) and runs inside something called a terminal. The terminal is just an interface/application that runs the shell.

Please refer to this Stack Exchange thread for a comprehensive description of the differences between the terminal and the shell.

In a nut-shell (no pun intended), the shell allows you to communicate with the operating system using text. The old school way!

Types of Shells

Every computer has some sort of shell and some have multiple shells. The syntax between them can be different, but fundamentally they carry out the same processes.

The most well known shell is the Bourne Again Shell (Bash) which is typically the default for Unix-like systems such as Linux and MacOS. However, MacOS recently changed its default to the Z-Shell in 2019, which is just an extension of Bash, and so has much of the same syntax and functionality.

Basic Commands And Functionality

Accessing The Shell

If you are on a MacOS system you can access the shell through the terminal application and when you boot it up it should look like this:

There are a few features you need to be aware with:

  • At the top bar you can see the zsh which indicates we are in the Z Shell
  • The egorhowell@Egors-MBP means I am logged into the user as egorhowell on Egors-MBP (Macbook Pro)
  • The % sign is the command prompt for the Z Shell. For the Bash shell it will be a dollar sign $.

Finding, Changing and Making Directories

We can use the shell to navigate through our computer like you would with the GUI of

  • The command ls (list) lets you see all the current folders and directories from where you currently are
  • We can see what directory we are in through pwd (print working directory)
  • We can create a directory by invoking the mkdir (make directory) command followed by the name of the directory we want to create
  • The command cd (change directory) followed by the directory you want to go into. You can also use cd .. to go back to the previous directory

Lets go through an example. We will view our current directory using ls and pwd , then create the directory _’egortest’ using mkdir and move into it with cd.

Notice how it now says ‘egor_test’ before the command prompt to indicate what directory we are in.

Creating, Editing and Viewing Files

Like directories, we can also create, edit and view files from the shell:

  • touch allows us to create a given file
  • echo allows us to write data and variables to the screen or to files
  • We can append to the file using ‘>>’ and overwrite using ‘>’
  • We can view the file using the cat command

Again, let’s go through a simple example. We will create a file called ‘test.txt’, then add the string ‘hello’ to it and view the file to make sure it is there. Then, we will overwrite it with ‘hello world’ and check it has indeed overwritten the file. Finally, we will append ‘hello world round 2’:

Other Useful Commands

  • The man command gives you the manual for a command
  • curl (client URL) lets you transfer data to your shell from some server which is useful when you want install packages such as Homebrew or Poetry
  • chmod (change mode) lets you change the permission and privileges for files
  • cp (copy) to copy files
  • rm (remove) to remove files and directories
  • mv (move) to move and rename files

This is just scratching the surface of the available commands and capabilities of the shell. I have linked here a full list of all the commands for some lovely night time reading!

What’s Happening Under The Hood?

When we type a command into the shell, if it is not inherent to shell, it looks through something called PATH. This contains directories where the shell searches through to see if it can match an exectuable related to your command. We can view the directories the shell consults by typing echo $PATH:

Note: I have the Anaconda distribution installed so my PATH might look slightly different to yours.

So when we type in a command, the shell looks through the above directories, in their given order, to try to find the binary/file to execute the command we inputted.

We can check where the file that is executed by the shell is through running the which command:

Here we see that the executable of mkdir is in /bin/mkdir and then we call that executable file to make a new directory called ‘egor’.

Purpose For Data Scientists

You may be wondering, when will we use the shell as Data Scientists? Well we may not use it as much as software engineers, but there are plenty of instances:

  • Managing project python dependencies using packages such as pyenv and asdf for production code
  • Installing tools to improve your Coding experience such as Homebrew
  • Downloading packages using Anaconda and pip

There also many other times in your Data Science career when you need to use the shell to achieve something!

Conclusion

In this post we went over the Shell is and some of its basic functionality. It is a very powerful tool and odds are you will use it at some point in your career. Therefore, it is worth getting comfortable with!

Another Thing!

I have a free newsletter, Dishing the Data, where I share weekly tips for becoming a better Data Scientist. There is no "fluff" or "clickbait," just pure actionable insights from a practicing Data Scientist.

Dishing The Data | Egor Howell | Substack

Connect With Me!


Related Articles