News: News

Data management in microscopy: outcomes of the ACCS project

Progress in technique and instrumentation in microscopy has paved the way for unprecedented scientific advances. The ever-increasing data volumes produced, the greater acquisition speed for data capture, the increasing requirements in storage and computing power for data processing as well as the lack of standardisation in data formats and metadata have however posed challenges for microscopy research facilities and researchers to organise and manage their workflows from data capture through to storage.

Microscopy Australia, in partnership with the Australian Research Data Commons (ARDC), The University of Queensland, Monash University, the University of Wollongong and AARNet, have developed a range of tools and guidelines to address these challenges. The work was completed under the Australian Characterisation Commons at Scale (ACCS) project (Work Package “Big-data electron and correlative microscopy from instrument to publication”).

The publications describing the outcomes of this work are listed below:

1. The landscape in informatics and data management at microscopy research facilities:
  • Poger, D., van Schyndel, J., Nguyen, H., Silver, J., & Goscinski, W. J. (2021). Orchestration and management of data generated by big-data electron microscopy instruments: A Discovery report. Zenodo. 
  • Poger, D., Yen, L., & Braet, F. (2023). Big data in contemporary electron microscopy: challenges and opportunities in data transfer, compute and management. Histochemistry and Cell Biology, Advance online publication. 
2. A higher-level service to transfer data:
3. Optimisation and automation of workflows for data movement:
4. Reproducible baselining and benchmarking of file-transfer performance:
5. Real-time data processing in cryo-electron microscopy:
6. Automation of repetitive tasks in data processing and analysis:
7. Making EM software more accessible to users:
  • van Schyndel, J. (2022). The Electron Microscopy Data Processing Portal: A new capability for data-intensive research. Zenodo.
  • The Electron Microscopy Data Processing Portal.
  • Motamen, S., van Schyndel, J., Villa, J., & Poger, D. (March 2023). Electron microscopy data processing portal. Australian Microscopy & Microanalysis Newsletter (Issue 157), 36
8. A data-management software for the big-data EM community:
9. Guidelines for data retention and disposal:
10. Guidelines for metadata collection:

August 23, 2023