The worldโ€™s leading publication for data science, AI, and ML professionals.

Pandas & Python Tricks for Data Science & Data Analysis – Part 3

This is the third part of my Pandas & Python Tricks

Photo by Andrew Neel on Unsplash
Photo by Andrew Neel on Unsplash

Introduction

A couple of days ago, I shared some Python and Pandas tricks to help Data Analysts and Data Scientists quickly learn new valuable concepts that they might not be aware of. This is also part of the collection of tricks I share daily on LinkedIn.

Pandas

Replace values from a dataframe based on conditions

If you want to replace values from a dataframe based on conditions

โœ… You can use the built-in ๐—บ๐—ฎ๐˜€๐—ธ() function from Pandas.

Below is an illustration ๐Ÿ’ก

Apply colors to your Pandas dataframe

Have you ever wanted to quickly find some information JUST by looking at your dataframeโ“

Things like:

โœจ Which values are negative in each column?

โœจ What is the maximum or the minimum value of each column?

โœจ Which values are below or above the average?

The list goes on and on…

A great way of viewing such information is by using colors ๐ŸŽจ

โœ… ๐—ฃ๐—ฎ๐—ป๐—ฑ๐—ฎ๐˜€.๐˜€๐˜๐˜†๐—น๐—ฒ is a built-in module that provides a high-level interface for styling your dataframe.

Here is a video illustration!

Print Pandas dataframe in Markdown

It is always better to print your data frame in a way that makes it easier to understand.

โœ… One way of doing that is to render it in a markdown format using the .๐š๐š˜_๐š–๐šŠ๐š›๐š”๐š๐š˜๐š ๐š—() function.

Let me know in the comment which one is your favorite.

โœจ With Markdown โœ… or without markdown โŒ

SQL-like queries through dataframe

Pandas’ power can’t be explored enough in Data Science๐Ÿ’ป๐Ÿ“Š

As Data Analyst or Scientist, you might want to filter ๐Ÿ”Ž through your data to find relevant insights.

โœ… This can be achieved using the built-in ๐—พ๐˜‚๐—ฒ๐—ฟ๐˜†() function in Pandas.

It runs queries based on boolean expressions, as you would write a natural language sentence! ๐Ÿ’ฌ

Below is an illustration ๐Ÿš€

Transform Scikit Learn Processing to Pandas dataframe

If you have been taking a closer look ๐Ÿง at Scikit learn preprocessing module, you might have noticed that the underlying functions return a numpy array ๐Ÿ”ข .

This can make it difficult to keep track of the original names of the features in the data.

Wouldn’t it be nice to have a Pandas ๐Ÿผ dataframe instead without any additional lines of code to keep those features’ names?

โœ… This can be achieved using the ๐˜€๐—ฒ๐˜_๐—ผ๐˜‚๐˜๐—ฝ๐˜‚๐˜ API from the new version (1.2) of Scikit Learn.

Below is an illustration ๐Ÿ’ก

Extract periods from the Datetime column

Days, weeks, months, or quarters ๐Ÿ—“ , ….. Each one can play an important role depending on the tasks at hand.

โœ… With the ๐˜๐—ผ_๐—ฝ๐—ฒ๐—ฟ๐—ถ๐—ผ๐—ฑ() function, you can extract from the date column each of such relevant information.

Below is an illustration ๐Ÿ’ก

Python

Number of elements in a list

Still using loops ๐Ÿ” to determine how often each item occurs in a list?

Maybe there is a better and much more elegant Pythonic ๐Ÿ way!

โœ… You can use the ๐—–๐—ผ๐˜‚๐—ป๐˜๐—ฒ๐—ฟ class from Python to compute the counts of the elements in a list.

Below is an illustration ๐Ÿ’ก

Combine elements from multiple lists

Are you trying to aggregate elements from multiple lists?

โŒ Stop using ๐—ณ๐—ผ๐—ฟ loops ๐Ÿ” and adopt the following approach.

โœ… The Python built-in ๐˜‡๐—ถ๐—ฝ() function.

Below is an illustration ๐Ÿ’ก

Create multiple lists from aggregated elements

When trying to aggregate elements from multiple lists, the most elegant and Pythonic way is to use the built-in ๐˜‡๐—ถ๐—ฝ() function.

Now, what if you want to proceed the other way around: create multiple lists from those aggregated elementsโ“

โŒ Forget ๐—ณ๐—ผ๐—ฟ loops ๐Ÿ”

โœ… Just combine the ๐˜‡๐—ถ๐—ฝ() function with ๐—ฎ๐˜€๐˜๐—ฒ๐—ฟ๐—ถ๐˜€๐—ธ *

Below is an illustration ๐Ÿ’ก

I am a big fan of list comprehension

Don’t just code in Python like most people, take the shortcut and the most efficient approach.

Imagine that you want to create a list with only even numbers from an existing one. The most obvious idea is using a "for" loop. But the most elegant one is using a list comprehension, which is more compact, simpler, and easier to debug.

Below is an illustration ๐Ÿ’ก

Where there is list comprehension there is a dictionary comprehension

Similarly to list comprehension, it is also possible to create dictionary comprehension. It also provides the same benefits compared to list comprehension.

Let’s consider having a dictionary where the keys are the index and the values are the actual numbers from the original list with the constraint of being an even number.

Below is an illustration ๐Ÿ’ก

Conclusion

Thank you for reading! ๐ŸŽ‰ ๐Ÿพ

I hope you found this list of Python and Pandas tricks helpful! Keep an eye on here, because the content will be maintained with more tricks on a daily basis.

Also, If you like reading my stories and wish to support my writing, consider becoming a Medium member. With a $ 5-a-month commitment, you unlock unlimited access to stories on Medium.

Would you like to buy me a coffee โ˜•๏ธ? โ†’ Here you go!

Feel free to follow me on Medium, Twitter, and YouTube, or say Hi on LinkedIn. It is always a pleasure to discuss AI, ML, Data Science, NLP, and MLOps stuff!

Before you leave find the last two parts of this series below:

Pandas & Python Tricks for Data Science & Data Analysis – Part 1

Pandas & Python Tricks for Data Science & Data Analysis – Part 2


Related Articles