Making Sense of Big Data

Reducing memory usage in pandas with smaller datatypes

Optimizing pandas memory usage by the effective use of datatypes

Parul Pandey
Towards Data Science
6 min readMar 15, 2021

--

Photo by Tolga Ulkan on Unsplash

Managing large datasets with pandas is a pretty common issue. As a result, a lot of libraries and tools have been developed to ease that pain. Take, for instance, the pydatatable library mentioned…

--

--

Principal Data Scientist @H2O.ai | Author of Machine Learning for High-Risk Applications