https://peertube.watch/videos/watch/956eb2dc-cce4-4692-81d4-7f8a8577b4b1 Workflow management tools have long been used in scientific computing to organise and operate workflows. Many such tools, e.g., Snakemake, Luigi, and Toil, have grown from the foundation of Make (wherein users define simple rules with interdependent inputs and outputs), incorporating additional features to suit increasingly complex user needs. Initially seeing a widespread uptake in bioinformatics, workflow managers have become commonplace in many fields, for example, high-energy physics (HEP). Analyses in HEP typically consist of many non-trivially related processes with widely varying requirements. Workflow managers can vastly simplify such analyses, providing user-friendly methods to define, review and run analysis workflows. Snakemake has emerged as a leading workflow manager for HEP, with an established user base spread across major experiments. Dialogue between developers and HEP has led to integrations for distributed storage/transfer frameworks, e.g., XRootD, FTP and Amazon S3, and scheduling frameworks, e.g., HTCondor, Slurm, and DRMAA. These integrations enable analysts to better leverage the distributed computing resources made available by experiments, significantly improving the efficiency of HEP analyses. Further collaboration between analysts and developers has seen Snakemake form the core of several standardised analysis frameworks aimed at improving analysis reproducibility such as REANA. This contribution discusses the current use of workflow managers in HEP, including best practices for their application. Additionally, the anticipated requirements of analysts are considered within the context of ever-increasing data scales in HEP. Speaker: Jamie Gooding, Track: Open Research devroom, Room: UB4.132, When: Saturday 3 February 2024, 17:45–18:00 (Brussels time), Schedule: FOSDEM 2024 event page, Slides & links Slides, HSF guide on Snakemake for analysis, Snakemake homepage, Licence: Creative Commons Attribution 2.0 Belgium (CC BY 2.0 BE) (as stated in the FOSDEM video archive). Attribution: “Workflow managers in high-energy physics: enhancing analyses with Snakemake” by Jamie Gooding, FOSDEM 2024. Original recording: 2024/ub4132/fosdem-2024-3415-workflow-managers-in-high-energy-physics-enhancing-analyses-with-snakemake.av1.webm from the FOSDEM video archive. This channel is an unofficial mirror; FOSDEM is a free and non-commercial event organised by the community for the community. Sat, 24 Oct 2026 01:30:46 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://peertube.watch https://peertube.watch/lazy-static/avatars/3b284627-b75f-409e-ac2e-10f4c2946f43.png https://peertube.watch/videos/watch/956eb2dc-cce4-4692-81d4-7f8a8577b4b1 All rights reserved, unless otherwise specified in the terms specified at https://peertube.watch/about and potential licenses granted by each content's rightholder.