Integrated Institute-Wide Data Acquisition and Analysis at FHI: Bluesky
At the beginning of the new decade, all data acquisition and analysis systems developed by the PP&B IT group will be compatible with the "bluesky" framework. This software was developed at NSLS-II and is another collaborative development in the EPICS [1] environment.
PP&B has been developing all data acquisition systems with the EPICS framework (http://epics-controls.org/) for many years. Since 2019 this development is also intensively supported by the MPG: https://epics.mpg.de. “Up to a point, EPICS makes all devices look the same. You can speak a common language to EPICS in the same way you can speak a common language to different websites. It’s the equivalent of the ‘http’ in a web address, but for hardware control.” (Dan Allan, NSLS-II). Bluesky stands on the shoulders of EPICS, and provides additional capabilities such as live visualization and data processing tools, and can export data into nearly any file format in real time. Bluesky was developed using “Python,” a common programming language that will make Bluesky simple for future scientists to modify, and to implement at new experiments.
The Bluesky software libraries developed and amongst others deployed at NSLS-II and APS facilitate collaboration between experiments, support streaming “on-the-fly” visualization and processing of large datasets, automate metadata capture for use in search queries or analysis, and integrate well with analysis software via the open-source scientific Python ecosystem. This enables scientists to leverage existing python programms at every stage of the experiment from data acquistion through data analysis. Through the integration of our compute cluster, machine learning algorithms can also be used for online optimization. Engagement from a wider user and developer community is beginning and will be actively supported (also with BESSY-II). The project is thoroughly documented (https://blueskyproject.io/) and the code is available under a standard open-source license.
Due to the manageable size of the FHI and its high flexibility, we are able to use the latest developments in this field and thus achieve a continuous object-oriented installation by using EPICS-7 with pvAccess. An in-house S3 storage system is provided by a first test installation. By submitting a BAR application at the beginning of the year, we are trying to expand this storage concept so that we can also meet the requirements e.g. of modern electron microscopy in the future.