Parallel batch processing in Python
Process in batches using joblib and show progress with tqdm
Published in
8 min readDec 27, 2021
Joblib is a great tool for parallelization but sometimes it is better to process the workload in batches and not in the default iterative way. In this article I’ll show:
- Standard way to parallelize using joblib and tqdm
- Why and when it does not work