message = message def notify (self, context): # Send notification here, below is an example title = f "Task failed" send_message(title, self. Users can build notification logic from a new base object, and call it directly from their DAG files.įor example, a notifier class might look something like this: from import BaseNotifier from my_provider import send_message class MyNotifier ( BaseNotifier ): template_fields = ( "message" ,) def _init_ (self, message): self. Notifiers introduce a new object in Airflow, designed to be an extensible layer for adding notifications to DAGs. Additionally, these functions would often require re-implementing an integration to a system (Slack, email, teams, etc.) that Airflow already had a pre-built integration (Provider) for, but couldn’t intuitively be used in the scope of that callback. Historically, these callbacks were generally written in an ad-hoc way. This parameter is a natural place to include custom logic that users require for sending notifications to email, Slack, or other systems of their choice about the state of their DAG. Notifiers: A New, Extendible Way to Add NotificationsĪirflow’s on_*_callback parameter allows DAGs to execute a Python callable based on the state of a DAG (i.e. The logs for the triggerer would previously be outside of the task logs, resulting in users needing to switch between views when debugging tasks built with deferrable operators.Īirflow 2.6 brings the triggerer logs into the UI, right alongside the task logs, resulting in a much cleaner debugging experience.
The triggerer monitors tasks that have entered the deferred state, and thus vacated their worker slot, resulting in cost savings and more efficient infrastructure utilization. Consolidated Trigger Logs in the UIĪirflow 2.2 introduced deferrable tasks and added a new component to Airflow, the triggerer. It would be remiss not to mention that in addition to the 2.6 release, the community also put out a new providers release which contains, among other things, an official Kafka Provider.
It also coincides with another milestone for the project – surpassing 30K Github stars.
Quantitatively, it contains over 500 commits from over 130 contributors, adding up to 35 new features, 50 general improvements, and 27 bug fixes. This release is a milestone for the community. This release comes just a few months after Airflow 2.5 was released, and both build upon features started in previous releases, while also introducing brand new concepts. Import Banner from ”./././components/Banner.astro” Īpache Airflow 2.6 was released earlier this week.
0 kommentar(er)
