https://peertube.watch/videos/watch/7c334795-9743-4cef-b3a8-4d21571b040e Efficient and well managed Orchestrating your Data processing Pipelines is crucial to provide the back-bone of the modern Data processing needs. With all the new rage of LLMS and AI, it's more important than ever to make sure that you can wrap your head around all the old, new and upcoming data processing tools and services you use, but also track the provenance, lineage of the data, and make it easy to author and mange those for distributed teams. This talk will bring to the light how modern Airflow 2.7+ provides proven and also modernised, easy nd nice to use ways on how to do it. Quite often you hear about the "new" orchestrator that aims to solve your orchestration needs. You can also often hear how it compares to Airlfow. However those comparisions often overlook the fact that since Airflow 2.0 has been introduced, it continues to evolve and piece-by-piece modernize itself and respond to the needs of processing even more data, and interacting with even more systems that were not even existing a year ago (think LLMs) - while continue harnessing the powers of other tools you already used in the past and bind them all together. New UI, New ways of writing your orchestration tasks, new ways to test them, tracking lineage, simpler authoring and intracting with object storages. And the comparision often overlook that if you start your journey with Airflow today, your experience will be quite a bit different than even 2 years ago (what usually most comparisions talk about). This talk will highlight some of the important ones. Also you might be surprised but this is all happening without breaking compatibility - hence you can still you use it in the "old way". But maybe it's time to learn new ways? Speaker: Jarek Potiuk, Track: Python Devroom devroom, Room: UD2.218A, When: Sunday 4 February 2024, 10:00–10:25 (Brussels time), Schedule: FOSDEM 2024 event page, Slides & links Apache Airflow Github Project, Apache Software Foundation, Presentation, Licence: Creative Commons Attribution 2.0 Belgium (CC BY 2.0 BE) (as stated in the FOSDEM video archive). Attribution: “`New` Workflow Orchestrator in town: "Apache Airflow 2.x"” by Jarek Potiuk, FOSDEM 2024. Original recording: 2024/ud2218a/fosdem-2024-1652--new-workflow-orchestrator-in-town-apache-airflow-2-x-.av1.webm from the FOSDEM video archive. This channel is an unofficial mirror; FOSDEM is a free and non-commercial event organised by the community for the community. Sun, 25 Oct 2026 01:58:17 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://peertube.watch https://peertube.watch/lazy-static/avatars/3b284627-b75f-409e-ac2e-10f4c2946f43.png https://peertube.watch/videos/watch/7c334795-9743-4cef-b3a8-4d21571b040e All rights reserved, unless otherwise specified in the terms specified at https://peertube.watch/about and potential licenses granted by each content's rightholder.