Apache Airflow for Data Science — How to Run Tasks in Parallel

Make multiple GET requests in parallel with Apache Airflow and Python

Dario Radečić
Towards Data Science
6 min readMar 10, 2022

--

Photo by Waldemar Brandt on Unsplash

In the previous article, we’ve configured Apache Airflow in such a way that it can run tasks in parallel. To do so, we had to switch the underlying metadata database from SQLite to Postgres, and also change the executor from Sequential to…

--

--