On the Fly and On the Flight: Scientific Data Analysis Beyond the Beamline

At synchrotron facilities, researchers use high-energy X-rays to uncover the structure and chemistry of materials — from batteries to biological systems. These experiments generate vast amounts of complex, high-dimensional data, and scientists need flexible tools, interactive exploration, and smart automation to make sense of it all. In this talk, we present a Python-based data processing suite designed to support X-ray spectroscopy workflows: from generating immediate data products to inform critical decisions during limited beamtime, to in-depth reprocessing off-site — or even offline.

The suite is built around a processing library that uses xarray for intuitive multi-dimensional data handling and provides reproducible, transparent analysis pipelines via command-line tools and Jupyter notebooks for coding-affine users. It also includes a PyQt-based desktop GUI that enables domain scientists to interactively explore their data and fine-tune processing steps without the need for prior programming experience.

We'll explore how Python libraries like xarray, pyqtgraph, and typer take the processing of raw experimental data from low-level wrangling in multiple dimensions to high-level exploration in intuitive interfaces. By providing different levels of balance between automation and user control we enable scientists with all levels of programming experience to create insightful datasets — whether they're at the beamline, back at their home institution, or already on the plane to the next conference.