
This quirk is specific to Apache Airflow, and it’s important to remember - especially if you’re using default variables and macros. This happens because Airflow can’t ensure that all of the data from 2:00 PM - 3:00 PM is present until the end of that hourly interval. An hourly DAG, for example, will execute its 2:00 PM run when the clock strikes 3:00 PM.

#AIRFLOW DAG LOGGING UPGRADE#
We strongly encourage your team to upgrade to Airflow 2.x. Note: Following the Airflow 2.0 release in December of 2020, the open-source project has addressed a significant number of pain points commonly reported by users running previous versions. Whether you’re new to Airflow or an experienced user, check out this list of common errors and some corresponding fixes to consider.


In an effort to provide best practices and expand on existing resources, our team at Astronomer has collected some of the most common issues we see Airflow users face. It’s an incredibly flexible tool that powers mission-critical projects, from machine learning model training to traditional ETL at scale, for startups and Fortune 50 teams alike.Īirflow’s breadth and extensibility, however, can make it challenging to adopt - especially for those looking for guidance beyond day-one operations. Streamline your data pipeline workflow and unleash your productivity, without the hassle of managing Airflow.Īpache Airflow is the industry standard for workflow orchestration.
