Towards data-driven next-generation transmission electron microscopy, SR Spurgeon, C Ophus, L Jones, A Petford-Long, SV Kalinin, MJ Olszta, …, Nature materials 20 (3), 274-279
Abstract
From its inception nearly a century ago, transmission electron microscopy (TEM) has emerged as a cornerstone of characterization in materials science, chemistry, physics and medicine1. TEM provides rich, directly resolved information about the structure and dynamics of phenomena spanning atoms to micrometres that are of great fundamental and practical significance to society. It has played a key role in protein and drug discovery2, redefined our understanding of crystalline solids3 and catalysed the electronics revolution that gave rise to today’s massively interconnected world4.
In spite of these numerous successes, many grand materials challenges remain outside of our present capabilities. Mastery of quantum phenomena, for example, requires insight into subtle and dilute electronic perturbations that can only be probed through sensitive multi-modal analyses closely linked to theory. Control of chemical reaction pathways in catalysts depends on access to interchangeable, finely tuned environments and the cumulative knowledge of a large library of prior experiments. True combinatorial engineering of high-entropy alloys demands on-the-fly experimental decision-making based on automated characterization. In these domains and more, a reimagined microscopy paradigm is needed to unlock entirely new classes of materials and functionality.
As in many other areas of science5, advances in TEM instrumentation now permit the rapid generation of vast data sets across a range of modalities, in which important connections might be more easily overlooked. Counterintuitively, microscopists focus on the methods already familiar to them, rather than harnessing more suitable tools from the full suite at their disposal. This situation is compounded by the growing complexity and closed-source nature of modern microscopes, which limit our ability and motivation to fully understand and customize their operation. Due to these barriers, the much-lauded promise of artificial intelligence (AI) and machine learning (ML) to revolutionize TEM experiment design, execution and analysis has not yet been realized. In contrast, other fields such as X-ray crystallography that have adopted open, standardized methods and data exchanges have witnessed enormous success6. Automated X-ray experimentation is now routinely conducted at scale, aided by easily accessible libraries of past work to plan and interpret future studies. Additive manufacturing is another area in which shared repositories of blueprints and techniques have empowered end users to conduct experimentation never imagined by their original creators. In electron microscopy, the growth of single-particle cryo-imaging demonstrates the untapped potential of automated ‘big data’ tools7 to transform our understanding of metals, semiconductors, ceramics and more.
Sweeping changes precipitated by recent technological innovations and the growth of modern data science tools call for a re-examination of the electron microscopy framework, shown in Fig. 1. This framework aims to discover knowledge about an unknown materials structure or process, employing a priori assumptions and an array of microscopy tools to probe different features of the unknown system. These features can then be distilled into salient physical mechanisms and quantifiable metrics through the eyes of various scientific disciplines. While there are many ways to define this framework, we broadly divide it into three overlapping categories: experiment design, feature extraction and knowledge discovery. These generally applicable categories provide a basis to understand the present state-of-the-art and its shortcomings, with a focus here on the application of data techniques to the physical sciences. In particular, we argue that an open, highly integrated and data-driven framework will transform characterization in the next-generation transmission electron microscope, benefitting both the physical and the biological sciences.
