CRYO-ELECTRON MICROSCOPY FOR THE
STUDY OF HYBRID MATERIALS AND
BIOLOGICAL SPECIMENS
A Dissertation
Presented to the Faculty of the Graduate School
of Cornell University
in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy
by
Katherine Anne Spoth
December 2018
© 2018 Katherine Anne Spoth. Cryo-Electron Microscopy for the Study of
Hybrid Materials and Biological Specimens is made available under a Creative
Commons Attribution-NonCommercial-ShareAlike 4.0 License:
https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode
CRYO-ELECTRON MICROSCOPY FOR THE STUDY OF HYBRID
MATERIALS AND BIOLOGICAL SPECIMENS
Katherine Anne Spoth, Ph.D.
Cornell University 2018
Cryo-electron microscopy has grown overwhelmingly in popularity in re-
cent years. Instrumental developments enable higher- and higher-resolution
studies of biological molecules to near-atomic resolution using single parti-
cle analysis, while cryo-TEM tomography gives three-dimensional information
about cellular structure.
This thesis applies cryo-EM in less-conventional studies: in applications to
fields outside of biology, and studies of cryo-scanning TEM (STEM) techniques
with new detector technology.
Vitrification methods for cryo-TEM perfectly preserve native structure in
solution. It is used here to study the early formation processes of meso-
porous silica nanoparticles, which are templated by self-assembling surfactant
molecules in solution. Direct imaging of this hybrid inorganic/organic mate-
rial with organics intact would be difficult without vitrifying the reaction solu-
tion. Through cryo-TEM imaging of different stages of the formation process
we identify synthesis parameters affecting the properties of the fully-formed
nanoparticles: relative reactant concentrations, stirring rate, and types of silica
precursors. We study formation of hierarchical structures of single-pore par-
ticles, from their initial formation to alignment in a 1D cylinder, higher-order
nanosheet, and helical structures. Structural preservation by cryo-EM is also
utilized to quantitatively describe a shape change in hexagonal silica particles
using cryo-STEM imaging, including three dimensional structure determined
using cryo-STEM tomography.
STEM has been the technique of choice for high-resolution imaging of ma-
terials, however conventional detectors discard much of the incident electron
dose making the technique’s use difficult for radiation-sensitive cryo-EM spec-
imens. A new direct detector for STEM, the Electron Microscope Pixel Array
Detector (EMPAD) records the full diffraction pattern at each scan position, al-
lowing nearly all electrons incident on a specimen to be used for imaging. In
particular, we describe a new technique for bright field imaging, where pixe-
lated detection over the central diffraction disk allows collection of many im-
ages at different relative tilts to the optical axis. Because the images are detected
separately, we can measure the tilt-induced image shift at each pixel and cor-
rect for the shifts, creating a coherent image with improved signal-to-noise ratio
and resolution over both BF-STEM and conventional TEM imaging. The im-
provement is greater for thick specimens, for which effects due to chromatic
aberrations are greatly reduced in STEM compared to TEM. At very low doses,
tcBF-STEM outperforms TEM, which will lead to improvements in tomography
where the total dose must be fractioned over all frames in the tilt series.
BIOGRAPHICAL SKETCH
Katherine A. Spoth was born in Buffalo, NY to John and Jeanne Spoth, the
second of four siblings.
Katherine began her scientific career at the University at Buffalo, graduat-
ing with a Bachelor’s of Science in physics and mathematics in 2012. During
her time at UB, she researched thermomagnetic stimulation of cells in the bio-
physics lab of Dr. Arnd Pralle. Her first experience at Cornell was a summer
project at CHESS, where she worked with Dr. Peter Revesz on portable X-ray
fluorescence spectroscopy in 2010. She returned to Cornell in Fall of 2012 to be-
gin her graduate work in Applied Physics, and joined the research group of Pro-
fessor Lena Kourkoutis that winter. After crashing the vacuum in the BioTwin
dozens of times, plunging hundreds of samples, and calculating thousands of
FFTs, she received the PhD in the Fall of 2018.
iii
ACKNOWLEDGEMENTS
Thank you, first and foremost to my adviser, Lena Kourkoutis, for your help
and guidance over my PhD. From teaching me everything about cryo-EM, to
finding exciting projects, and to getting my talks in shape, I am extremely grate-
ful for all you’ve done for me. Thank you to David Muller for serving on my
special committee and for the opportunity to collaborate on the cryo-STEM EM-
PAD project. Thank you Warren Zipfel, for reminding me that other types of
microscopes exist and for serving on my special committee.
To the “microscope whisperers” who keep everything up and running and
can always find the beam: John Grazul, Mick Thomas, and Mariena Silvestry-
Ramos—I can’t wait to learn more from all of you.
Thank you to my labmates, who felt like a family—David Baek, Ben Sav-
itzky, Ismail El Baggari, Michael Zachman. To Jade Noble and Berit Goodge
for being badass ladies of science and great friends. To the new crew: Yue Yu,
Michelle Smeaton, Taylor Moon—if you break stuff, I’ll try to fix it!
To Lisa for all the breakfasts and the cat pictures.
To my family: Mom, Dad, Jim, Abbey, and Emily for your love and support
over the years. For visits, vacations, phone calls and group texts, thank you for
being there.
To Patrick, for driving to Ithaca approximately 70 times for visits and finally
for good. I’m so glad to be your wife.
iv
TABLE OF CONTENTS
Biographical Sketch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
List of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 Introduction 1
1.1 Introduction to electron microscopy . . . . . . . . . . . . . . . . . 1
1.1.1 The principle of reciprocity . . . . . . . . . . . . . . . . . . 3
1.1.2 Specimen thickness for EM . . . . . . . . . . . . . . . . . . 4
1.2 Cryo-electron microscopy . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Three-dimensional information from cryo-EM . . . . . . . 5
1.2.2 Specimen preparation by vitrification and plunge freezing 7
1.2.3 Specimen handling and imaging at low temperature . . . 10
1.2.4 Radiation sensitivity and low-dose imaging . . . . . . . . 11
1.2.5 Contrast and SNR in cryo-TEM images . . . . . . . . . . . 12
1.2.6 Cryo-STEM . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.3 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2 Early formation of quasicrystalline mesoporous silica nanoparticles
imaged using cryo-TEM 21
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2.1 Specimen preparation . . . . . . . . . . . . . . . . . . . . . 25
2.2.2 Cryo-TEM imaging . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.3 Micelle size measurement . . . . . . . . . . . . . . . . . . . 27
2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 Cryo-TEM imaging of micelles . . . . . . . . . . . . . . . . 30
2.3.2 Effect of pore-expander concentration . . . . . . . . . . . . 32
2.3.3 Effect of stirring . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
v
3 Cryo-TEM characterization of single-pore silica nanoparticle forma-
tion 36
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Shape-change in hexagonal mesoporous silica nanoparticles imaged
with cryo-STEM 46
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2.1 Specimen preparation . . . . . . . . . . . . . . . . . . . . . 47
4.2.2 Cryo-EM imaging . . . . . . . . . . . . . . . . . . . . . . . 48
4.2.3 Cryo-STEM tomography . . . . . . . . . . . . . . . . . . . . 50
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3.1 Cryo-STEM imaging of MSNs . . . . . . . . . . . . . . . . . 51
4.3.2 Imaging shape change using cryo-STEM . . . . . . . . . . 53
4.3.3 Shape change directed by solvent used for synthesis . . . . 57
4.3.4 Three-dimensional structure of MSNs . . . . . . . . . . . . 58
4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5 Biological cryo-STEM imaging using the electron microscope pixel ar-
ray detector 61
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2 The electron microscope pixel array detector . . . . . . . . . . . . 62
5.3 Imaging biological specimens with the EMPAD . . . . . . . . . . . 65
5.3.1 Classes of biological specimens . . . . . . . . . . . . . . . . 65
5.4 Tilt-corrected bright field STEM imaging . . . . . . . . . . . . . . . 69
5.4.1 The contrast transfer function in tcBF-STEM . . . . . . . . 73
5.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.5.1 Comparison of tcBF-STEM, conventional TEM, and zero-
loss EFTEM for imaging E. coli . . . . . . . . . . . . . . . . 77
5.5.2 Tolerable electron dose for tcBF-STEM . . . . . . . . . . . . 81
5.6 tcBF-STEM in thin specimens . . . . . . . . . . . . . . . . . . . . . 83
5.7 Possibilities for electron ptychography in biological specimens . . 86
5.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
vi
6 Conclusions 90
A Practical aspects of low dose imaging with the EMPAD 93
A.1 Choice of convergence angle . . . . . . . . . . . . . . . . . . . . . . 93
A.2 Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
A.3 Operating at low dose . . . . . . . . . . . . . . . . . . . . . . . . . 95
A.3.1 Setting and measuring dose rate . . . . . . . . . . . . . . . 95
A.3.2 Focusing and finding your specimen . . . . . . . . . . . . . 96
A.3.3 Defining areas for focus and acquisition . . . . . . . . . . . 97
A.3.4 Choosing an exposure time . . . . . . . . . . . . . . . . . . 98
A.4 Considerations for tomography . . . . . . . . . . . . . . . . . . . . 98
B PAD data analysis using Python 100
References 119
vii
LIST OF FIGURES
1.1 Diagrams depicting TEM and STEM. . . . . . . . . . . . . . . . . 3
1.2 Illustration of the principle of reciprocity for TEM and STEM. . . 4
1.3 Basic workflow of single particle cryo-EM. . . . . . . . . . . . . . 6
1.4 Illustration of electron tomography, in which a specimen is im-
aged at a range of tilts, the images aligned, and reconstructed to
give a 3D structure of the specimen. . . . . . . . . . . . . . . . . . 7
1.5 Illustration of crystalline ice formation vs. vitrification. . . . . . . 8
1.6 The plunging process for specimen vitrification. . . . . . . . . . . 9
1.7 Cryo-TEM images of chromatophore vesicles in Rhodobacter
sphaeroides at increasing dose rate. . . . . . . . . . . . . . . . . . . 12
1.8 CTFs plotted for TEM imaging. . . . . . . . . . . . . . . . . . . . . 14
1.9 CTF for HAADF-STEM imaging. . . . . . . . . . . . . . . . . . . . 19
2.1 Illustrated formation process of mesoporous silica nanoparticles. 22
2.2 Quasicrystalline mesoporous silica nanoparticle showing do-
decagonal tiling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3 Drift correction of 0.25-second CCD frames. . . . . . . . . . . . . 27
2.4 Optimal threshold picking for the Laplacian-of-Gaussian blob
detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5 Cryo-TEM image showing micelles prepared with 100 µL TMB. . 29
2.6 Effect of different synthesis conditions on the final structure of
MSNs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.7 Cryo-TEM images of organic micelles. . . . . . . . . . . . . . . . . 31
2.8 Skew-normal curves fit to the micelle size distributions from four
initial concentrations of pore-expander (TMB). . . . . . . . . . . . 33
2.9 TEM images of particles dried at the end of synthesis. . . . . . . 34
2.10 Size distribution and fully-formed silica particles formed with
and without stirring. . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1 The relative concentrations of components in formation of
single-pore silica nanoparticles studied in this chapter. . . . . . . 39
3.2 Illustrations of single ring nanoparticle structure. . . . . . . . . . 40
3.3 Micelles imaged using cryo-TEM. . . . . . . . . . . . . . . . . . . 40
3.4 Single ring particles formed after the addition of TMOS. . . . . . 41
viii
3.5 Silica rings with PEG added show ordering into long, 1D assem-
blies in solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.6 Silica rings with PEG showing higher-order structure. . . . . . . 43
3.7 Evolution of the single-pore silica nanoparticle system as addi-
tional TEOS is dosed to the particles. . . . . . . . . . . . . . . . . 44
4.1 A modified blotting technique for nanoparticles in ethanol. . . . 49
4.2 Projection images of hexagonal mesoporous silica nanoparticles
in 80:20 ethanol:water solution in two orientations. . . . . . . . . 52
4.3 Two sets of mesoporous silica particles imaged in solution, after
vacuum drying, and after exposure to humidity. . . . . . . . . . . 54
4.4 Histograms of shape change across 66 particles. . . . . . . . . . . 55
4.5 Decrease in particle length observed from cryo-STEM to
humidity-exposed state. . . . . . . . . . . . . . . . . . . . . . . . . 56
4.6 Comparison of MSNs synthesized in ethanol and in water. . . . . 57
4.7 Cryo-STEM tomography reconstruction of a vacuum-dried
mesoporous silica nanoparticle. . . . . . . . . . . . . . . . . . . . 59
5.1 Images of the electron microscope pixel array detector. . . . . . . 63
5.2 Coherent detection using the EMPAD. . . . . . . . . . . . . . . . . 64
5.3 STEM imaging of E. coli using the EMPAD. . . . . . . . . . . . . . 66
5.4 Method of tilt correction for efficient coherent bright field imaging. 71
5.5 The bright field phase (left) and amplitude (right) CTF for an
axial STEM detector, calculated for Cs = 1.2 mm and ∆ f = 10 nm. 75
5.6 Contrast transfer function of off-axis BF detectors. . . . . . . . . . 75
5.7 Images of the same cell taken using TEM, zero-loss EFTEM, and
tcBF-STEM at dose of 7.5 e/Å2. . . . . . . . . . . . . . . . . . . . . 78
5.8 Fourier ring correlations of thick and thin regions of the images
in Figure 5.7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.9 tcBF-STEM compared to TEM and zero-loss EFTEM at a dose of
0.5 e−/Å2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.10 Tilt-corrected BF-STEM images collected sequentially with in-
creasing dose rates per frame. . . . . . . . . . . . . . . . . . . . . 83
5.11 Scattering profiles in thin ribosome and thick E. coli specimens
calculated using data from the EMPAD. . . . . . . . . . . . . . . . 84
5.12 tcBF-STEM imaging of ribosomes using the EMPAD. . . . . . . . 85
ix
5.13 tcBF-STEM shifts calculated in x, y for E. coli specimen and ribo-
somes under different imaging conditions. . . . . . . . . . . . . . 85
5.14 Bright field ptychography compared to tcBF-STEM for thin and
thick biological specimens. . . . . . . . . . . . . . . . . . . . . . . 88
x
LIST OF ABBREVIATIONS
ADF Annular dark field
AEAPTMS Aminopropyltrimethoxysilane
BF Bright field
CBED Convergent beam electron diffraction
CCD Charge-coupled device
CMOS Complimentary metal oxide semiconductor
CTAB Hexadecyltrimethylammonium bromide
CTF Contrast transfer function
DOF Depth of field
EELS Electron energy loss spectroscopy
EFTEM Energy filtered transmission electron microscopy
EM Electron microscopy
EMPAD Electron microscope pixel array detector
FIB Focused ion beam
FRC Fourier ring correlation
FFT Fast Fourier transform
HAADF High-angle annular dark field
MAPS Monolithic active pixel sensor
MSN Mesoporous silica nanoparticles
PEG Polyethylene glycol
PSF Point spread function
SIRT Simultaneous iterative reconstruction technique
SNR Signal-to-noise ratio
ss-MSN Shape-shifting mesoporous silica nanoparticles
STEM Scanning transmission electron microscopy
tcBF-STEM Tilt-corrected bright field scanning transmission electron microscopy
TEM Transmission electron microscopy
TEOS Tetraethyl orthosilicate
TMB Mesitylene
TMOS Tetramethyl orthosilicate
WDD Wigner distribution deconvolution
xi
LIST OF SYMBOLS
α Semiconvergence angle
∆ f Defocus
λ Wavelength
χ Aberration function
Cs Spherical aberration coefficient
d Probe size
k Spatial frequency
t̃ Contrast transfer function
xii
CHAPTER 1
INTRODUCTION
Microscopy is a technique beautiful in its simplicity, allowing scientists to di-
rectly observe—to look at—structures of interest. Electron microscopy reveals
structures down to the atomic scale, and is used across fields to image nanoscale
features with high resolution. This work is a study of cryo-electron microscopy
(cryo-EM), in which the material observed is maintained in a film of vitrified,
glass-like ice to preserve structures as they are in a liquid environment but sta-
ble in the high vacuum environment of the electron microscope. This technique
is valuable to many types of materials—most well-known in biology for studies
of cellular ultrastructure [1, 2, 3] and high-resolution structural determination
of macromolecules [4], but also has been applied to materials science for struc-
tural preservation of soft materials [5], imaging electronic phases existing at
low temperature [6, 7], accessing interfaces between solids and liquids [8], and
maintaining both organic and inorganic components in hybrid materials [9, 10].
In this thesis, we apply cryo-EM imaging techniques to a new system: meso-
porous silica nanoparticles whose formation is dictated by the interaction of
organic and inorganic components in solution. We also demonstrate new cryo-
scanning transmission electron microscopy (STEM) techniques with direct de-
tection technology allowing for dose-efficient STEM imaging of biological spec-
imens.
1.1 Introduction to electron microscopy
Electron microscopes operate under similar principles to optical microscopy, but
use electrons to image. Electrons have a much smaller wavelength than visible
1
light and therefore allow much higher resolutions. The wavelength of a 300 kV
electron is 1.97 pm, compared to photons in the visible spectral range where the
minimum wavelength is on the order of 400 nm. The electron microscope was
invented by Ernst Ruska and Max Knoll in 1931, and shortly after that surpassed
the resolution of light microscopes [11]. Ruska won the Nobel Prize for his
invention in 1986.
The basic operation of an electron microscope is illustrated in Figure 1.1. The
two parts show conventional TEM (left) and scanning TEM (STEM, right). In
both methods, electrons from the source are accelerated through a high voltage
(typically 60-300 keV) then focused and manipulated by magnetic lenses. In a
TEM, the beam arrives at the sample in a wide, parallel form, interacts with
the specimen, and then is focused and magnified by the objective lens. The
image is collected using a 2-dimensional detector, either a scintillator-coupled
CCD or a direct electron detector. In STEM, the beam is focused to a small
probe (a demagnified image of the source) in the sample plane by the objective
lens above the specimen. The beam scatters from the specimen, and different
types of detectors located at different scattering angles post-specimen can be
used to form an image. Typical geometries are illustrated in Figure 1.1 and
include an annular detector which collects elastically scattered electrons at high
angles (a high-angle annular dark field, or HAADF, detector), and a small, on-
axis detector collecting a bright field (BF) image.
2
Figure 1.1: Diagram of TEM (left) and STEM (right) operation.
1.1.1 The principle of reciprocity
As we utilize both TEM and STEM in this thesis, and sometimes atypical modes
of each, it is useful to have a method to relate the two techniques. We can un-
derstand STEM techniques in terms of TEM and vice-versa by noting that in
the microscope if we reverse the ray paths and switch the position of the source
and detector, we transition from TEM to STEM (or STEM to TEM). This is called
the principle of reciprocity. The simplest example is the equivalence of conven-
tional TEM with on-axis BF-STEM imaging using a point detector, illustrated in
3
Figure 1.2: Illustration of the principle of reciprocity for TEM and STEM.
Figure reproduced from [12].
Figure 1.2, reproduced from [12].
1.1.2 Specimen thickness for EM
Both STEM and TEM are transmission microscopy methods—the beam of elec-
trons must pass through the specimen for any information to be collected. Be-
cause of this, the specimen thickness is extremely important to the methods and
the results obtained. In general, images of thin specimens are much more sim-
ple to interpret due to their lack of obfuscating multiple scattering events. For
this reason most specimen preparation methods for EM are designed to create
specimens as thin as possible.
For cryo-EM, which is discussed next, we consider scattering from vitreous
4
ice. Here, the mean free path, a parameter which describes the average path
an electron travels before it scatters, is roughly 300 nm at 300 keV for inelastic
scattering [13].
1.2 Cryo-electron microscopy
Cryo-EM was developed in the biological sciences to allow for imaging biolog-
ical specimens in the high vacuum of the EM without disturbing the natural
structure of these materials. The typical approach for stabilizing material for
observation in the EM had in the past been plastic-embedding, heavy metal
staining, and physical slicing (microtomy) for cells and tissues, and negative
staining of macromolecules. While these techniques allow for high-resolution
observation of biological specimens, the techniques used to preserve and thin
the specimens create artifacts, potentially altering the structures of interest [14].
Staining processes additionally limit the achievable resolution because it is the
stain particles, not the structure of interest itself, that is imaged.
1.2.1 Three-dimensional information from cryo-EM
Today, the term cryo-EM has become in some fields nearly synonymous with
single particle cryo-EM: a technique illustrated in Figure 1.3 in which thousands
of purified biological molecules are imaged in projection. Because each particle
orients differently (ideally randomly) on the grid, the three-dimensional struc-
ture of the particle can be reconstructed by finding two-dimensional classes and
determining the orientation of each.
5
Figure 1.3: Basic workflow of single particle cryo-EM. Reproduced from
[15].
Another route to 3D information, suitable for larger assemblies as well as
whole cells is cryo-electron tomography. The process is illustrated in Figure 1.4.
Here, one specimen is imaged at a range of tilts as large as permitted by the
holder and microscope geometry, the images are aligned and reconstructed to
reveal the three-dimensional architecture of the specimen.
Together, these methods won the 2017 Nobel Prize in Chemistry [17].
6
Figure 1.4: Illustration of electron tomography, in which a specimen is im-
aged at a range of tilts, the images aligned, and reconstructed
to give a 3D structure of the specimen. Reproduced from [16].
1.2.2 Specimen preparation by vitrification and plunge freez-
ing
Vitrification is the process of freezing a liquid into an amorphous, glass-like
state. Freezing at extremely high cooling rates (104 K/s) or at high pressure
suppresses formation of crystalline ice, allowing vitrification to occur. Crys-
talline ice has the potential to damage structures of interest, so care must be
taken with the cooling process. This is illustrated in Figure 1.5. However, with
the formation of vitreous ice, the structures of interest are perfectly preserved.
The possibility of freezing specimens for EM dates back to the 1960s in the
work of Fernandez-Moran, who showed the first electron micrographs of ice
crystals [19]. The first cryo-EM of biological specimens was demonstrated by
Taylor and Glaeser the following decade [20]. A protocol for routine plunge-
7
Figure 1.5: Illustration of crystalline ice formation vs. vitrification. (a) The
structure of liquid water in a specimen. (b) Water expands as it
slowly freezes into crystalline ice, damaging the specimen. (c)
Vitrification, by rapid cooling or high pressure freezing, creates
amorphous ice in the same configuration as the original water
molecules. Reproduced from [18].
freezing was published by Dubochet et. al. [21], which is the method of choice
for vitrification of biological specimens less than 1 micron thick (including all
specimens presented in this thesis). For even thicker specimens (up to 10 µm),
vitrification can be achieved using high pressure freezing [22], however, for ob-
servation at high resolution in the electron microscope, specimens of this size
will require additional thinning through cryo-microtomy or cryo-FIB.
The process of vitrification by plunge freezing is illustrated in Figure 1.6.
First, holey carbon films on copper grids are made hydrophilic by plasma clean-
ing in a mixture of oxygen and argon gas for 8–10 s. Alternatively, we use a glow
discharger (GloQube, Electron Microscopy Sciences) which uses air to create the
plasma. A hydrophilic surface is necessary to properly wet the grid to create a
thin film of suspension in the holes, and prevent beading up of the specimen.
2-4 µL of specimen in aqueous suspension is pipetted onto the prepared
grids. The specimen droplet is then blotted to a thin layer using filter paper,
usually for times from 2-8 seconds. In this thesis, we used a manual plunger
8
Figure 1.6: The plunging process for specimen vitrification. (a) Photo of
the manual plunger used in this thesis. (b) Specimen sus-
pended in aqueous solution is pipetted onto the TEM grid. (c)
Filter paper is used to blot the specimen, creating a thin film of
liquid on the grid. (d) The specimen is rapidly frozen by plung-
ing into the cryogen. Graphics courtesy Michael Zachman.
where the blotting is done by hand. Increasingly common are robotic vitrifi-
cation systems, which can improve reproducibility by blotting with exactly the
same amount of force for the same amount of time1. These systems have the
additional advantage of controllable temperature and humidity, which reduces
evaporation of the specimen film in the crucial seconds before plunging into
cryogen. With a manual system, this is often accomplished using a foot pedal,
so that as soon as the filter paper is removed the specimen is plunged into the
cryogen.
To achieve the high cooling rates needed for vitrification, we use a mixture of
37% ethane and 63% propane liquified in a reservoir cooled by liquid nitrogen.
Use of liquid nitrogen alone does not allow vitrification because of the Leiden-
frost effect in which the boiling nitrogen forms an insulating layer around the
1The author has found, however, that “practice makes perfect” with regards to getting con-
sistent results even with the manual plunger.
9
specimen and greatly slows the cooling rate. The mixture of ethane and propane
is useful because it remains liquid at liquid nitrogen temperature [23]. Though
pure ethane and pure propane are often used, both are solid at nitrogen temper-
ature and require heating.
1.2.3 Specimen handling and imaging at low temperature
Specimens not only need to be cooled rapidly but must be maintained at low
temperature to prevent devitrification, a phase transition from vitreous to crys-
talline ice that occurs around -140◦C. To accomplish this, grids are stored in liq-
uid nitrogen from the time of plunge-freezing until their transfer to the electron
microscope.
Transfer and imaging below this devitrification temperature are accom-
plished here using a side-entry cryo-holder. The holder contains a liquid nitro-
gen dewar which cools the specimen. An accessory loading station submerges
the holder’s tip in a reservoir of liquid nitrogen for loading grids without ex-
posure to air. Transfer to the microscope is done as rapidly as possible, with
a small metal shutter covering the grid to slow crystallization of water vapor
onto the specimen. The microscope airlock is pre-pumped so that the sample is
in vacuum as rapidly as possible.
Even though the microscope column is at high vacuum, some water vapor
molecules are still present inside. This is the reason a cold finger is commonly
used, to trap these molecules on a cold piece of metal inside the column. For
cryo-EM imaging, the cold specimen can have the unwanted effect of attracting
this water vapor. This is prevented by the use of cryo-blades, nitrogen-cooled
10
metal pieces designed to sit close to the specimen and capture the water vapor
instead, preventing a buildup of amorphous ice on the surface. The microscopes
used here are equipped with retractable cryo-blades for this purpose.
1.2.4 Radiation sensitivity and low-dose imaging
Vitrified specimens, and biological materials in general, are extremely dose-
sensitive. Imaging at low temperatures slows the rate of ionization damage
[24] but the overall dose tolerance is low. For high resolution imaging, doses are
typically kept below 20 e−/Å2 [25]. The ultimate limit to resolution in cryo-EM
is the radiation tolerance of the specimen.
The effect of dose on image quality is illustrated in Figure 1.7, from reference
[15]. Images of vitrified chromatophore vesicles show low contrast and signal
to noise at low dose rates. It’s important to note that for electron tomography,
the total dose must be fractionated over the number of tilts—in an individual
frame, a dose of 1-2 e−/Å2 is typically used.
In order to limit the dose in areas of interest, a common practice for cryo-
EM imaging is to focus in an area adjacent to the area chosen for the exposure,
so that the first time an intense beam of electrons hits the area, data is being
collected. This process has been automated for TEM imaging and is known as
“Low-Dose Mode” [26]. These methods were used for all the TEM imaging in
this thesis2.
2The TEM data in Chapter 4 was taken before Low Dose was installed on any of the mi-
croscopes, and the author mimicked the conditions by focusing, moving the stage 1 µm, then
recording the image
11
Figure 1.7: Cryo-TEM images of chromatophore vesicles in Rhodobacter
sphaeroides at increasing dose rate. Challenges of imaging at
low dose are illustrated, namely low contrast and SNR. Figure
reproduced from [15].
1.2.5 Contrast and SNR in cryo-TEM images
Due to the radiation sensitivity of cryo-specimens limiting the total tolerable
dose, images taken with cryo-TEM suffer from low contrast and signal-to-noise
ratio (SNR). Contrast in biological/organic material is low because the elements
contained in the material are all low in atomic number, as well as similar in
weight to the surrounding vitreous ice.
The contrast transfer function in TEM
To understand the contrast in TEM images, we utilize the linear image approx-
imation, which states that for a weak-phase object (a very thin specimen), the
image contrast is described by the convolution of the object function with the
12
point spread function (PSF) of the microscope.The Fourier transfer of the PSF is
the contrast transfer function (CTF) which describes the amount of each spatial
frequency transferred to the final image. This is a useful framework by which to
understand the effects of aberrations in the microscope, as well as the benefits
and weaknesses of different imaging modes.
Contrast transfer functions for TEM imaging are shown in Figure 1.8. Here,
parameters used are of our Titan TEM, with voltage set to 300 kV, spherical aber-
ration (Cs) of 1.2 mm, and variable defocus shown in the different traces. Figure
1.8(a) shows an undamped CTF; Figure 1.8(b) takes into account the damping
envelope function which depends on energy spread, defocus spread, and an-
gular spread of the illumination (basically, deviations from perfect coherence in
illumination).
The TEM contrast transfer function is oscillatory, which to quote Dr. Kirk-
land “is about the worst thing that can happen” [12]. Here, some spatial fre-
quencies are transmitted as bright, some as dark, and a few, where the CTF
crosses zero, not at all.
To increase contrast in low-dose cryo-TEM images a few strategies are com-
monly used. First, an objective aperture is used, cutting off the CTF oscillations
at higher frequency, but imposing a resolution cutoff at the aperture diameter.
Secondly, the microscope is defocused, often by several microns. As we see in
Figure 1.8, this improves the contrast at low spatial frequencies, but moves the
first zero crossing to lower frequency. The ability to move the zero crossings
is exploited in single particle analysis, in which images are acquired at a few
defoci and the CTF divided out in Fourier space for each particle to fill in in-
formation at the zero crossings for a single defocus. This CTF-correction step is
13
Figure 1.8: CTFs plotted for TEM imaging. (a) CTF with no damping func-
tions. (b) CTF taking into account partial coherence. Parame-
ters used are: 300 kV beam energy, 1.2 mm spherical aberration,
70 micron objective aperture cutoff, defocus as indicated in the
legend, defocus spread 100 Å, and angular spread in illumina-
tion 0.5 mrad. The code used is by Earl Kirkland [12], which
as published does not take into account the energy spread, but
illustrates the intensity falloff well.
required for high-resolution reconstructions.
Effects in thick specimens and energy filtering
For specimens thicker than the inelastic mean free path ( 300 nm for 300 kV elec-
trons), image quality is further degraded by the presence of inelastic scattering.
Electrons which lose energy in the specimen are focused by the objective lens to
different planes, known as chromatic aberration. This leads to a blurring in the
final image. Energy-filtered TEM (EFTEM) can be used to remove the electrons
which lose energy in the specimen by placing a slit around the zero-loss peak,
but any electrons filtered out have only contributed to damage in the specimen
and not at all to the image. This effect can be severe in very thick specimens as
14
will be discussed in Chapter 5.
SNR improvements through direct detection
In the past decade, huge resolution improvements have been made possible
in cryo-TEM due to the advent of direct electron detectors. In terms of noise,
the performance of a detector is quantified using a metric called the detective
quantum efficiency (DQE), a function of spatial frequency, k, which is given in
Equation 1.1. The DQE describes the amount of noise the detector adds to an
image.
SNR2
DQE out
(k)
=
SNR2
(1.1)
in (k)
SNRout and SNRin are the SNR of the output and input of the detector,
respectively.
Until the advent of direct detection, the highest DQE available at high spatial
frequency was by recording on photographic film [27]. CCDs use a scintillator
to convert the electron signal into photons, a step which adds noise—leading to
DQEs worse than film despite the convenience they offer of electronic collection
[28].
Direct detectors for TEM imaging are now commonplace, and a requirement
for high-resolution single-particle structural determination. Modern direct de-
tectors are monolithic active pixel sensors (MAPS) detectors fabricated using
complimentary metal oxide semiconductor (CMOS) technology. Electrons in-
teract directly with the sensor, eliminating the need for a scintillator. The fast
15
readout rate of new detectors allows them to operate in electron counting mode,
where under low flux the detector counts and localizes single electron events,
which greatly reduces noise/increases DQE [28]. Additionally, the detectors can
greatly lessen the effects of specimen drift by operating in movie mode. Sub-
sequent alignment of fast frames restores the high-resolution information, un-
derscoring the importance of cross-correlation methods for low-SNR EM data
[29, 30].
1.2.6 Cryo-STEM
History of STEM imaging in biology
Current imaging techniques for biological specimens center almost exclusively
on imaging with TEM. However, the earliest studies with the STEM were fo-
cused on its applications in biology [31, 32]. The simultaneous detection of
multiple signals provided good dose-efficiency and allowed the computational
combination of dark- and bright-field signals [32, 33]. Rapid development in bi-
ological STEM continued for the next decade, with applications in both stained
[34, 35] and unstained biological specimens [36, 37] as well as individual macro-
molecules [38, 39, 40]. However, the achievable resolution of STEM far exceeded
the detail allowed by fixation and staining techniques, and the advent of cryo-
TEM shifted the focus of the field to this technique [21]. STEM remained impor-
tant for quantitative mass-mapping using the dark-field mode [41, 35] and for
microanalysis [42, 43].
The first STEM imaging of a frozen-hydrated specimen was demonstrated
by Leapman et al. in 1991 [44]. Dark-field imaging was possible, but did not
16
produce as much detail as phase-contrast cryo-TEM images. Cryo-STEM has
also been investigated as a tool for micoanalysis of frozen-hydrated specimens
though the use of electron energy loss spectroscopy (EELS) [45, 46]. However,
both imaging and spectroscopy suffer due to the stringent dose limitations of
cryogenic imaging.
STEM has more recently been recognized as a technique superior to TEM
in analysis of thick specimens due to the far lessened contribution of chromatic
aberration compared to TEM [47]. Here, bright-field detection is the method of
choice due to the reduction in beam-broadening in thick specimens compared
to the dark-field mode [48].
Some interest in cryo-STEM has therefore reemerged, with demonstrations
of cryo-STEM tomography in whole cells showing results comparable to cryo-
TEM imaging [49]. Additionally, dark-field imaging has been used in some
cases to detect heavier biological components in whole cells [50] and even in
single-particle analysis of ferritin [51], in which Z-contrast in dark field STEM
was used to localize the heavier iron atom in the molecular structure.
Despite this renewed interest, biological STEM has been mostly left by the
wayside in favor of popular TEM techniques. The ease of producing high-
contrast images of frozen-hydrated specimens through phase contrast imaging
in TEM is likely the cause of the first departures of the field from STEM imag-
ing, with its continuing momentum a result of the rapid pace of higher- and
higher-resolution results in single-particle analysis bolstered by the advent of
direct detectors for transmission electron microscopy [52].
17
STEM imaging
Figure 1.1 shows an illustration of the electron microscope operating in STEM
mode and the conventional types of detectors used: an axial detector collecting
the bright field (BF) signal and an annular detector which collects electrons that
scatter elastically to high angles. Depending on the collection angle, this is a
high angle annular dark field (HAADF), or more generally annular dark field
(ADF) detector. We recall from Section 1.1.1 that the CTF for BF-STEM with
a small detector is identical to that for conventional TEM by the principle of
reciprocity.
Imaging with an ADF detector is an incoherent method and leads to a very
different transfer function, shown in Figure 1.9. This imaging method has a
monotonic transfer function, decreasing from one to zero with increasing spatial
frequency. Because of the simplicity of the CTF, images taken using ADF-STEM
are much more easily interpretable than BF methods (STEM or TEM). We ex-
ploit this in Chapter 4 to measure shape changes in silica nanoparticles, useful
because we don’t need to consider magnification changes or diffraction effects
from defocusing when measuring distances in the images. We also use STEM-
tomography to reconstruct the three-dimensional structure of the particles.
HAADF-STEM is a Z-contrast method—that is, atoms with higher atomic
number scatter more strongly into the high angle region so intensity here is di-
rectly related to the atomic number. For this reason, it is seldom used in biology
where all the materials of interest have similar Z-number. For the silica mate-
rial however, there is good contrast between the particles and the surrounding
solution.
18
Figure 1.9: CTF for HAADF-STEM imaging. Here, we are at 300 keV, 1.2
mm Cs, 550 Å defocus, using a 10 mrad semicovergence angle.
1.3 Thesis outline
In the following two chapters, we apply conventional cryo-TEM techniques to
a new type of system: formation of hybrid organic-inorganic mesoporous sil-
ica nanoparticles. The formation of these particles takes place completely in
solution, so we utilize the ability of cryo-TEM to directly image the particles’
components without distortion due to drying.
In Chapter 4, we use cryo-ADF-STEM imaging to quantify shape change in
mesoporous silica particles that shift from hexagonal to six-pointed-star upon
exposure to humidity in the environment. Again, the ability to observe the par-
ticles directly in solution allows us to verify the true structure and more care-
fully investigate the shape changes that take place upon drying and exposure to
humidity.
Chapter 5 describes new breakthroughs in cryo-STEM imaging made pos-
sible by the invention of the Electron Microscope Pixel Array Detector (EM-
PAD), a direct detector for STEM which promises a revolution of its own kind
19
by allowing collection of the full diffraction pattern at each STEM scan pixel,
allowing collection of STEM images in cryo-specimens without discarding any
incident dose. We discuss general practices for imaging with the detector, then
a new method for coherent bright-field imaging using the entire central disk
called tilt-corrected BF-STEM (tcBF-STEM), comparing to TEM results.
20
CHAPTER 2
EARLY FORMATION OF QUASICRYSTALLINE MESOPOROUS SILICA
NANOPARTICLES IMAGED USING CRYO-TEM
Cryo-TEM imaging is used to directly observe solution-driven formation
processes in mesoporous silica nanoparticles. These materials consist of in-
organic silica which is deposited onto an organic template. Without preser-
vation by vitrification, the organic components are impossible to visualize.
Here, we use cryo-TEM to study size distributions of the organic micelles,
which comprise a hydrophobic pore-expander material surrounded by surfac-
tant molecules. By changing the relative concentration of pore-expander and
surfactant, and by changing the stirring rate, we identify conditions leading to
desired final silica pore structure. We find that a quasicrystalline pore struc-
ture is possible with a wide distribution of micelle sizes, requiring both a high
concentration of pore expander and stirring to fully incorporate the expander
material. This work gives new insights into both quasicrystal formation and
structural control in mesoporous silica.
2.1 Introduction
Mesoporous silica is a silica material with pores in the 2-50 nm size range. These
materials, first developed in the early 1990s [53], are ideal for applications in
sensing and catalysis due to their highly-ordered interconnected pore structure
and large surface area [54].
Mesoporous silica nanoparticles (MSNs) are synthesized in a solution-driven
process, illustrated in Figure 2.1 [55]. The pore structure is dictated by template
21
Figure 2.1: Illustrated formation process of mesoporous silica nanoparti-
cles. Surfactant molecules self-assemble in solution and added
silica precursors condense onto the surfaces, forming the meso-
porous particles [55].
molecules, usually surfactants, which self-assemble in solution due to their am-
phiphilic nature—that is, with a hydrophobic end and a hydrophilic one, the
surfactants assemble such that the hydrophobic ends resist contact with the re-
action solution. Following that, a silica precursor added to the solution con-
denses onto the template and forms the material. Many methods for control-
ling and precisely modulating the synthesis have been studied, and factors to
control include silica sources, surfactant types, the reactant solution’s composi-
tion and pH, rate and duration of stirring, and temperature [56]. By controlling
the structures of these materials, they can be tailored more exactly to different
applications. However, the formation processes are not easily understood as
they take place completely in solution. Cryo-EM is an ideal technique for this
analysis because the formation can be studied directly in the synthesis solution,
without changing any of the particles’ properties during the drying process.
22
Several nanoparticle systems have been studied with cryo-EM previously
not only to confirm structure without artifacts that could be induced by drying
or fixation [57, 58, 59, 60, 61] but also to study solution-dependent processes
such as particle formation [62, 63, 64, 65]. In a process similar to this work, one
study looked at nucleation of silica particles without the presence of any tem-
plating molecules to determine the mechanisms of that formation, finding that
small “primary particles” form and subsequently aggregate as silica particles
grow in solution [66].
Cryo-EM not only allows imaging the particles’ formation in solution, it also
preserves the organic components that can be destroyed or distorted in the dry-
ing process. By imaging in native solution with the organics intact, we gain
new insights into the formation processes of these materials. To better under-
stand how variables in the formation process influence the final form of MSNs,
we study a model system which yields quasicrystalline nanoparticles. The qua-
sicrystallinity here can be described by alternating patterns of squares and trian-
gles between adjacent pores. Figure 2.2a shows a typical structure of the parti-
cles with the square-triangle tiling illustrated in Figure 2.2b. To learn more about
how quasicrystallinity occurs both in silica nanoparticles and in other materials,
we use the mesoporous nanoparticle formation pathway as a model system and
study the components under different relative concentrations as well as the ef-
fect of stirring on the particles’ structures.
The variation in pore sizes required for quasicrystal formation can be
achieved by adding a hydrophobic pore-expander molecule, which can mod-
ify the resulting micelle size. Without this, one type of surfactant molecule
should always assemble into micelles of a consistent size, leading to particles
23
Figure 2.2: Quasicrystalline mesoporous silica nanoparticle showing do-
decagonal tiling. (a) Dry-state TEM image of the fully-formed
particle, scale bar 100 nm. The FFT, inset, shows the 12-fold
symmetry. (b) The particle’s quasicrystalline tiling, illustrated
by a repeating pattern of squares and triangles connecting pore
centers. Coloring represents the six directions possible in do-
decagonal space.
with uniformly sized pores. In the work here, hexadecyltrimethylammonium
bromide (CTAB) is the surfactant molecule used to form the micelles. Without
the presence of other components, MSNs formed from CTAB and silica precur-
sors have a hexagonal shape and uniform 6-fold symmetry of identical cylindri-
cal pores. To form quasicrystalline nanoparticles we added mesitylene (TMB), a
hydrophobic compound which becomes encapsulated by the CTAB molecules
and acts to expand the micelle and therefore the pore sizes. With the addi-
tion of TMB quasicrystalline particle formation has been observed. The exact
mechanism and required conditions for quasicrystalline formation are difficult
to study as they occur in solution and on the nanoscale.
Using cryo-TEM, we directly imaged the micelles snap-frozen in solution.
Both the micelle-forming surfactant CTAB and the pore-expander TMB consist
24
of light, organic materials that are not stable outside of the solution. Drying
the micelles on a grid for simple observation in the TEM would destroy the
structures; if the micelles were visible at all, drying processes would certainly
deform their shape. Instead, we preserve a snapshot of the formation process by
vitrifying the micelles in native solution to view their structure and interaction
in the early stages of synthesis.
To understand how micelle sizes control crystallinity and how different prac-
tices in this early stage of formation affect the final structure we captured the
micelles in solution prepared under a range of conditions. Two key variables
studied were stirring time of the surfactant and pore expander and the relative
concentration of each component in solution.
2.2 Methods
2.2.1 Specimen preparation
To probe the formation processes of quasicrystalline MSNs, we observed the
synthesis solution under different conditions using cryo-TEM. The parameters
varied were relative concentration of the pore expander, [TMB], as well as the
stirring rate of the solution with constant [TMB]. In all cases the basic sample
preparation procedure was the same.
Samples for cryo-TEM analysis were prepared by plunge freezing. 4 µL of so-
lution were pipetted onto a Quantifoil holey carbon TEM grid that was plasma
cleaned for 10s with a mixture of argon and oxygen gases. The specimen was
25
manually blotted for 6-8 seconds and plunged into a liquefied mixture of ethane
and propane [23].
This preparation was repeated for all five conditions measured: 0mM, 4mM,
72mM and 116mM TMB added to CTAB/NH4OH solution with stirring at 600
RPM for two hours, as well as 72 mM TMB with stirring for only 30s to disperse
the TMB with the solution left static for two hours to match the stirred condi-
tions. An aliquot was then removed for plunge-freezing as described above,
then the silica precursors tetramethyl orthosilicate (TMOS) and aminopropy-
ltrimethoxysilane (AEAPTMS) were added to produce the MSNs corresponding
to the micelle conditions visualized with cryo-TEM.
2.2.2 Cryo-TEM imaging
Grids were transferred to the TEM and imaged below the recrystallization tem-
perature of the specimen using a Gatan cryo-transfer holder. Cryo-TEM images
were collected under low-dose conditions using a customized FEI Titan Themis
300 operating at 300 kV equipped with a cryo-box and FEI Ceta 16M camera.
All images were collected at -2 µm defocus at a dose between 80-120 electrons
per Å2. Consistency in the defocus condition is important, as changes in defocus
create small changes in magnification leading to differently perceived particle
sizes.
Images were collected using a quasi-movie mode on the CCD; the dose was
fractionated into 0.25s frames which were then aligned via cross-correlation.
Figure 2.3 illustrates the importance of this method even when imaging with a
CCD; in Figure 2.3a, the summed frames show significant drift relative to the
26
Figure 2.3: Drift correction of 0.25-second CCD frames. (a) Uncorrected,
summed frames mimic a single long exposure on the camera,
and show clear drift both in real space and in the asymmetry
in the FFT (inset). (b) Cross-correlating and aligning the frames
corrects much of the drift resulting in a higher-quality image.
aligned image in Figure 2.3b
2.2.3 Micelle size measurement
Micelle size distributions were determined from cryo-TEM images. Images
were cropped to regions of micelles suspended in vitreous ice in a TEM grid
hole. Using in-house Python code, micelles were located via a Laplacian-of-
Gaussian blob-detecting algorithm. For each image, the intensity threshold of
the blob-detecting algorithm was set by hand to ensure correct identification of
micelles without inclusion of noise. The specifics of this are illustrated in Figure
2.4.
27
Figure 2.4: Optimal threshold picking for the Laplacian-of-Gaussian blob
detection. Thresholds were set by hand for each image in
which micelles were measured to ensure picking most of the
particles while rejecting noise. The image shown here is the
same as in Figure 2.3, with contrast inverted for correct opera-
tion of the blob-finder. Detected blobs are denoted with red X,
here we see intensity thresholds that are too high (left), optimal
(center), and too low (right).
After determination of the micelle positions, their sizes were found to higher
accuracy than the blob-finder allows by fitting each “blob” to√a 2-D Gaussian in-√
tensity distribution.The diameters quoted are given by 2 2 σ2x + σ2y , where σx
and σy are the standard deviation of the 2D Gaussian in directions x, y deter-
mined by the longest and shortest axis of the elliptical distribution. Size distri-
butions for each condition were calculated from 5-7 images collected at different
locations across the TEM grid. Figure 2.5 shows the fit micelles on a small por-
tion of the image shown above, along with the histogram of all micelles sizes
from that preparation condition.
To describe the shape of the micelle size distributions, the histograms were
fit to a skew-normal distribution using nonlinear least squares fitting in Python.
Equation 2.1 shows the equation for this distribution.
28
Figure 2.5: Cryo-TEM image showing micelles prepared with 100 µL TMB.
White circles show the diameters found by the Gaussian inten-
sity fitting. The histogram, right, shows the size distribution of
all particles from this TMB concentration condition. It has been
fit to a skew normal distribution.
( ( ) ( ))
α (x − µ) (x − µ)2
A 1 + erf √ exp −
2 2σ2
(2.1)
σ
Here erf is the error function, A, µ and σ are amplitude, average size and
scale parameter, respectively. The parameter α describes the departure from the
normal distribution and improves fitting for distributions with a longer tail. The
skew-normal distribution was chosen because it describes the asymmetry of the
distribution created by the fact that there exists a minimum size for micelles, so
there is a tail of the size distribution contributed by the larger micelles [67].
29
2.3 Results
2.3.1 Cryo-TEM imaging of micelles
Cryo-TEM is valuable in these experiments because it allows us to directly ob-
serve the organic micelles which are impossible to observe with dry-state TEM
because they dissociate on the grids or blend in with the carbon. The resulting
fully-formed silica particles can be observed in dry state and allow us to de-
termine under which conditions quasicrystallinity occurs (Figure 2.6) [9]. Here
we see that the pH of the reaction solution controls the resulting particle size,
while varying the concentration of TMB regulates the pore structure and qua-
sicrystallinity. From the dry state we can tell what the key parameter is, but we
need a method to directly observe the micelles to understand how changing the
concentration changes the micelle structures and from there, the pore structures
of the silica NP.
With cryo-TEM we are able to directly image the organic micelle building
blocks. The particles as seen in cryo-TEM in vitrified ice over a hole in the grid
are shown in Figure 2.7. Three conditions are represented here—particles syn-
thesized with 4 mM TMB (a) and with 72 mM TMB (b) plus stirring, as well
as 72 mM TMB without stirring (c). The ability to directly observe the micelles
allows us to further probe their properties —here their size distribution —and
connect that to our observations of the final-stage silica particles. Observation
of these intermediate steps, impossible without cryo-TEM, leads to greater un-
derstanding of how the synthesis can be tailored to specific final structures.
30
Figure 2.6: Effect of different synthesis conditions on the final structure
of MSNs. In the array of figures, we see that increasing pH
(x-axis) causes an increase in particle size, while increasing
the concentration of pore-expander TMB (y-axis) leads to qua-
sicrystalline pore structure. Though we identify the parameter
which causes quasicrystallinity, without cryo-EM we cannot
directly observe the micelles to verify the mechanism. Scale
bar is 100 nm.
Figure 2.7: Micelles are clearly observable in vitrified solution in cryo-
TEM images. With the ability to directly observe these building
blocks of the MSNs we investigate the link between quasicrys-
talline pore structure and certain synthesis conditions, in this
work both TMB concentration and stirring rate.
31
2.3.2 Effect of pore-expander concentration
From the cryo-TEM images we have directly extracted the micelle size distri-
butions using the fitting procedure described in Methods. The results for four
different TMB concentrations: 0 mM, 4 mM, 72 mM, and 116 mM are shown in
Figure 2.8.
First, the distribution of CTAB-only micelles (no TMB) is shown. This dis-
tribution represents a bottom limit for the micelle sizes, as a sphere of CTAB
molecules is the smallest possible micelle. With increasing TMB the most obvi-
ous change is a shift in the peak position to higher diameter. This peak position
remains relatively constant for all cases with TMB, however with increasing
TMB we observe an increase in the width of the distribution. This corresponds
to quasicrystallinity in the final particles, shown in Figure 2.9.
2.3.3 Effect of stirring
In addition to the concentration of pore expander, we also examined the effect
of stirring on the micelle size distribution and final pore structure. The results
are shown in Figure 2.10.
We observe that stirring is a requirement for creating the broadened micelle
size distribution required for quasicrystallinity. The unstirred micelles (Figure
2.10, orange) show a narrower size distribution than those that were stirred (Fig-
ure 2.10, blue), with the normalized stirred distribution appearing wider with
more particles at larger diameters. It is apparent from the fully-formed silica
particles, inset, that the stirring here is the difference between particles with
32
Figure 2.8: Skew-normal curves fit to the micelle size distributions from
four initial concentrations of pore-expander (TMB). The con-
centration increases from top to bottom in the figure. Condi-
tions with TMB show a shifted peak position relative to CTAB-
only. The width and positive skewness of the distribution in-
creases with TMB concentration, suggesting a sizable popula-
tion of larger micelles is a prerequisite for quasicrystalline pore
structure.
33
Figure 2.9: TEM images of particles dried at the end of synthesis with
TMB concentrations corresponding to those in Figure 2.8. Only
the higher concentrations resulted in quasicrystalline structure.
Scale bars: 50 nm.
Figure 2.10: Size distribution and fully-formed silica particles formed with
72 mM of TMB both with (blue) and without (orange) stirring.
We observe that stirring is necessary to broaden the size dis-
tribution of micelles, and this in turn is a requirement for par-
ticles with quasicrystalline pore structure. Scale bars: 50 nm.
34
column-like pores and those with dodecagonal symmetry.
2.4 Conclusions
Cryo-TEM enabled us to directly observe hybrid organic/inorganic nanoparti-
cles in solution. We studied specifically the pore expander plus surfactant mi-
celles which template the pore structure. By determining their size distribution,
we learned the effect of changing reactant concentrations: a high concentra-
tion of pore expander is required to create size-disperse micelles which are a
requirement for quasicrystallinity. Furthermore, stirring is a necessary step to
create the wide range of size necessary for quasicrystalline formation. This work
gives insights not only into formation of mesoporous silica, but quasicrystalline
materials in general.
35
CHAPTER 3
CRYO-TEM CHARACTERIZATION OF SINGLE-PORE SILICA
NANOPARTICLE FORMATION
The formation steps of single-pore nanoparticles and higher-order structures
utilizing these ring particles as building blocks are explored using cryo-TEM.
We image formation steps, including organic pore-templating micelles alone,
micelles with silica ring particles, PEGylated rings that form highly-ordered,
one-dimensional chain structures in solution. Ring particles are dosed with ad-
ditional silica to explore if an ordered system or tube-shaped particle can be
synthesized by this method. Cryo-TEM is vital to this experiment as it preserves
both the organic and inorganic structures, showing the true structure of parti-
cles and superassemblies in solution. By comparing cryo- and dry-state TEM
we observe that drying processes artificially increase density of the particles,
obscuring information about their ordering.
3.1 Introduction
In the previous chapter, we used cryo-EM to directly observe formation pro-
cesses of complex structures in terms of very basic building blocks. Here, we
look more carefully into building blocks of a different structure: single-pore sil-
ica ring particles, which in turn assemble into cylinders, helices, and nanosheets.
We gain additional insights into chemistries of formation processes in porous
silica systems.
The system studied here is a simple ring-shaped nanoparticle just 10 nm in
diameter. These particles are interesting precisely due to their small size—small
36
enough for clearance from the body by the kidneys, they have many promis-
ing applications in nanomedicine [68]. The single-pore structure defines two
distinct faces of the particle which can be differently functionalized for drug-
delivery purposes. Precise control over formation processes in these types of
materials will allow their continued design for specific applications.
The formation in this system is controlled by the precise chemistry of the
formation process: relative concentrations of the component materials drive the
structure and properties of the resulting nanoparticles. Because all the structure-
determining interactions take place in solution, cryo-TEM is an ideal tool to
characterize these formation processes.
3.2 Methods
In this study, we image snapshots of the formation process of single-pore silica
nanoparticles. In each condition, we plunge-freeze the reaction solution on a
Quantifoil 2/1 holey carbon TEM grid that was plasma-cleaned for 8–10s using
a mixture of oxygen/argon gas in a Fischione plasma cleaner. 3–4
µ
L of solution was pipetted onto the grid, manually blotted for 2–6 seconds and
plunged into a mixture of liquified ethane-propane [23]. The specimen was
stored under liquid nitrogen and imaged at temperature around -170◦C using
a Gatan 626 cryo-transfer holder. Images were taken using both an FEI F20
equipped with cryo-blades (Gatan) and an FEI T12 BioTwin. For each image the
microscope used was specified. For all images, low-dose procedures for cryo-
TEM were implemented with separate areas used for focusing and exposure.
37
For dry-state images, the particles in solution were pipetted onto a continu-
ous carbon grid and allowed to dry.
We imaged several steps in the formation process: first, the organic pore tem-
plate in solution consisting of centrimonium bromide (CTAB) molecules sur-
rounding pore expander mesitylene (TMB), next after the addition of silica pre-
cursor tetramethyl orthosilicate (TMOS), then follow what happens under two
separate pathways: dosing of additional silica precursor tetraethyl orthosilicate
(TEOS), or application of polyethylene glycol (PEG) to the particles for stabiliza-
tion. In each case, cryo-TEM reveals details not discernible through dry-state
TEM, whether due to the loss of stabilization from the solution components or
merely artificial density increase through the drying process. We also observe
the results of dosing additional silica precursor to the system.
The conditions imaged here, without PEG, are summarized in Figure 3.1.
The table indicates the relative molar concentrations of each component. Base
values are 12.9 mM CTAB, 71.9 mM TMB, and 22.8 mM TMOS. The images seen
here correspond to other conditions described in the Results section: Figure
3.1(a) is pictured and described in Figure 3.4; Figure 3.1(b), (c), (d) are discussed
in Figure 3.7. Detailed information about the particle synthesis will be included
in the supplemental information in [69].
3.3 Results
The structure of the ring particles is shown in Figure 3.2. First, micelles con-
sisting of the surfactant CTAB surrounding hydrophobic TMB form in solution.
Here, the TMB is not incorporated to induce size dispersity of the micelles as
38
Figure 3.1: The relative concentrations of components in formation of
single-pore silica nanoparticles studied in this chapter.
we saw in the previous chapter. Instead, it increases the deformability of the
micelles which we theorize to be a requirement for ring formation [69]. The sil-
ica precursor, TMOS, is added to the solution and attaches to the micelles in a
ring shape. Finally, PEG is used to stabilize the ring structure. We utilize cryo-
EM to image each step in this process and to understand the specifics of how
the ring formation occurs.
The first stage we observe using cryo-TEM is the micelles in solution shown
in Figure 3.3. Upon inspection, micelles appear uniformly-sized with spheri-
cal shape. The ability to directly observe micelles is only possible using cryo-
EM—without the stabilizing presence of vitrified solution micelles would fall
apart and not be observable in the dry state.
The next step in the formation is addition of TMOS, the silica precursor.
39
Figure 3.2: Illustrations of single ring nanoparticle structure. (a) Each com-
ponent used in the synthesis. (b) The ring formation process,
in which silica primary particles formed form TMOS attach to
a TMB-swollen micelle in a ring shape [69].
Figure 3.3: Micelles imaged using cryo-TEM. In this region, spherical mi-
celles appear evenly distributed and uniformly sized. Micelles
were imaged at 120 kV using the BioTwin.
40
Figure 3.4: Single ring particles which form after the addition of TMOS.
Inset higher-magnification views of ring particles show detail
for front- and side-views. Inset field-of-view is 18 nm. Imaged
using the F20 at 200 kV.
Cryo-TEM images at this step (Figure 3.4) show the resulting particles: individ-
ual rings formed around the organic template. Since the particles are randomly
oriented and imaged in projection, we observe them from several angles. Insets
in Figure 3.4 show two interesting projections: front- and side-views.
In both the insets and the full image there are some noticeable properties of
the particles. Firstly, most rings appear to have some modulations in intensity
that suggest they are made up of silica particles 2 nm in diameter, which is con-
sistent with a model in which silica forms primary particles that in turn attach to
the micelles [66]. Secondly, we observe some rings forming dimers, suggesting
that these particles may have tendencies to form more complex, higher-ordered
structure.
After addition of PEG to the reaction solution, cryo-TEM images revealed
41
Figure 3.5: Silica rings with PEG added show ordering into long, 1D as-
semblies in solution. Rings line up with spacing 4.2 nm, as
shown in the inset FFT. Image taken at 120 kV using the T12
BioTwin.
extended cylindrical assemblies of the single-pore nanorings, shown in Figure
3.5. Upon the removal of surfactant and subsequent drying, PEG is intended
to stabilize the nanorings which appear as single particles in dry-state TEM.
Our observations with cryo-TEM however show that PEG promotes ordering
in the system. Further examples of this are observed in Figure 3.6, in which
we observe helical superassemblies of these cylinders in solution, as well as 2D
nanosheets which formed upon drying the cylinders in solution on a continuous
film.
Finally, we also observe drying as a pathway to ordered structures.
The cylinders of PEGylated NPs observed with cryo-TEM can give way to
nanosheets when dried upon a carbon support (Figure 3.6). Both the cylindri-
cal assemblies and dried nano sheets illustrate a path to formation of higher-
ordered structures through PEGylation of the silica rings.
42
Figure 3.6: Silica rings with PEG showing higher-order structure. (a) Im-
aged with cryo-TEM, the 1-D cylinders in turn form helical as-
semblies in solution. (b) Upon drying, the PEGylated rings can
form nanosheets on a carbon film.
We examined the possibility of an additional path to ordering: the addition
of another type of silica precursor, TEOS. This material hydrolyzes more slowly
than TMOS—so rather than forming particles in solution, it tends to condense
onto the already-existing ring particles. We imaged the ring formation system
after several additional doses of TEOS in both cryo- and dry-state TEM. The
results are shown in Figure 3.7. At low doses of TEOS, in cryo-TEM we see rings
starting to assemble into pairs and triples. In the dry-state images this ordering
is less apparent and the higher density of rings resulting from the drying process
obscures their interactions.
Diluting the particles for clearer dry-state TEM imaging appears to be an
obvious solution. However, in this type of synthesis, all of the particles’ proper-
ties derive from the relative concentration of reactants in the formation solution.
So, by diluting for sample preparation one would also be changing the relevant
chemistries. Because of this, cryo-TEM remains the most reliable way to observe
43
Figure 3.7: Evolution of the single-pore silica nanoparticle system as ad-
ditional TEOS is dosed to the particles (left-to-right). The top
and bottom images are cryo-TEM and dry-state images of the
same condition. We notice that upon drying, the particle den-
sity increases to a state in which the ordering cannot easily be
discerned. With the addition of TEOS, cryo-TEM shows parti-
cle chains of increasing length, but decreasing regularity. Aver-
ages of single rings oriented such that we look down the pore
axis show increase in diameter from 10 nm to 13 nm from the
first condition. Field-of-view of the inset averages is 20 nm.
Cryo-TEM images were collected using the F20 at 200 kV.
the system.
Upon dosing of additional TEOS we expected that the ordering may con-
tinue to increase, possibly leading to formation of a tube-shape particle. How-
ever, we see in the rightmost two panels of Figure 3.7 that the alignment instead
becomes less regular, with the striped alignment less apparent in images of these
concentrations. The clear, regular rings spacing that we see in the PEGylated
44
case (Figure 3.5) is not observed here.
As TEOS is added we also investigated the averaged effect on individual
ring structure by observing particles oriented face-on, such that we observe the
ring shape looking along the pore axis. From each condition, we aligned and
averaged particles in this projection to get an idea of the average shape. These
are shown in the insets in Figure 3.7. We observe an overall increase in diameter
as TEOS is dosed from 10 to 13 nm. In the fourth condition not enough single
rings were observed in the correct orientation to make the corresponding mea-
surement. This averaging technique is analogous to single-particle cryo-EM,
especially in the case of a 2D class-average. Single particle analysis has now
been applied to porous silica particles upon this suggestion [70].
3.4 Conclusions
We used cryo-TEM to elucidate the interactions between the components of
single-ring nanoparticles and understand their formation. We observed forma-
tion of a set of silica nanostructures: from the single-ring particles, to PEGylated
1D chains, to 2D sheets and 3D helices. Understanding the formation of these
structures allows a more targeted development of porous silica nanoparticles
in the future. By studying this system with cryo-EM we directly observed the
interaction of organic and inorganic components, avoiding the loss of organic
components and possibility of structural changes that occurs upon drying.
45
CHAPTER 4
SHAPE-CHANGE IN HEXAGONAL MESOPOROUS SILICA
NANOPARTICLES IMAGED WITH CRYO-STEM
Cryo-STEM imaging is used to fully characterize a shape change observed
in hexagonal mesoporous silica nanoparticles. We show ADF STEM’s utility for
imaging the silica nanoparticles against the lighter ice material. The nanopar-
ticles are imaged in solution, after drying in the microscope vacuum, and after
humid exposure in the laboratory. This experiment provided a complete de-
scription of the shape change in all dimensions of the particles, as well as a
stark illustration of the importance of specimen preparation for TEM analysis
and the possibility of drying artifacts. The three-dimensional structure of the
particles is confirmed using cryo-STEM tomography. The structure of the par-
ticles as well as data about the shape change are used to suggest a mechanism
behind the particles’ transition from hexagonal to six-angle star.
4.1 Introduction
Shape change materials are a class of materials that change their shape in re-
sponse to an external stimulus, for example, light, heat, or environmental hu-
midity [71, 72]. Here, we study a type of mesoporous silica nanoparticle that
undergoes an irreversible shape change in response to humidity in the environ-
ment, changing from hexagonal to star-shape [73]. Mesoporous silica particles
have been used in nanomedicine for drug encapsulation and delivery but with
the use of capping molecule to contain the relevant material [74]. The shape-
shifting MSNs (ss-MSN) studied here could release their cargo in response to
humidity in the environment, eliminating the need for more complicated sys-
46
tems.
To fully characterize the hexagonal ss-MSNs they must be studied in their
native state, easily preserved by cryo-EM. Since they change shape in response
to environmental factors, the specimens must be carefully handled to preserve
the particles’ properties. This is accomplished by imaging particles which have
been vitrified in their native solution, protecting the MSNs from environmental
humidity and drying-based structural changes.
The nanoparticles here are significantly larger than those characterized in the
previous chapters—around 200 nm across and with lengths around 300 nm. The
large size of these particles makes them amenable to imaging using ADF STEM
in which we observe good contrast between the relatively heavier silica material
and the lighter reaction solution. STEM imaging is also advantageous for this
case because we wish to perform quantitative size measurements of the parti-
cles, consistent across different images and conditions. Since ADF imaging is
done in focus, we do not have to consider slight defocus-induced magnification
changes that are common in TEM imaging. We also demonstrate ADF-STEM
tomography of these particles revealing their three-dimensional architecture.
4.2 Methods
4.2.1 Specimen preparation
Samples for cryo-TEM and cryo-STEM were prepared by plunge freezing. 3–4
µL of nanoparticle suspension were pipetted onto a 2/1 holey Quantifoil TEM
47
grid that had been plasma etched for 8–10 seconds (Fischione Model 1020
plasma cleaner with oxygen/argon gas) to ensure hydrophilicity. The grid was
blotted manually with filter paper to create a thin film of solution and quickly
plunged into a liquefied mixture of 37% ethane and 63% propane [23], vitrifying
the solution. Samples were stored under liquid nitrogen until observation in the
electron microscope.
Compared to specimens synthesized in water, those in pure ethanol were
more difficult to optimize to good, thin ice conditions for imaging. This is partly
due to the solvent wicking upwards between the tweezers after it is pipetted on
the grid and then being redeposited as the sample is plunged into the cryogen.
A modified blotting technique was used, in which one small piece of filter paper
was placed between the reverse-close tweezers to absorb the solvent that tended
to wick upward between the tweezer ends1. The specimen was blotted as usual
for up to 8 seconds and both filter paper pieces removed immediately before
plunging. With this method, particles in pure ethanol could be blotted to be
sufficiently thin for TEM imaging. The blotting method is illustrated in Figure
4.1.
4.2.2 Cryo-EM imaging
Cryo-TEM analysis was completed at 120 kV using an FEI Tecnai BioTWIN
TEM. Annular dark field cryo-STEM imaging was performed at 200 kV in an
FEI Tecnai F20 TEM/STEM equipped with cryo-blades. We used the typical 9.6
mrad semiconvergence angle for imaging and the conventional ADF detector.
1A different approach may be to use anti-capillary tweezers which have larger separation
between the two halves.
48
Figure 4.1: A modified blotting technique for nanoparticles in ethanol.
The solvent that wicks between the tweezers is removed by the
upper filter paper, which is then removed immediately before
plunging.
Vitrified samples were transferred to the STEM and imaged below -175◦C using
a Gatan 626 cryo-transfer holder.
Direct measurement of the shape change was performed using cryo-STEM of
particles synthesized in a mixture of 80% ethanol, 20% water. The modified blot-
ting scheme described above was used with these specimens. The sample first
was imaged in cryo-condition with the vitrified reaction solution intact. The
specimen was then vacuum dried inside the cryo-STEM which was achieved
by increasing the temperature of the cryo-holder until the surrounding vitri-
fied solution sublimated, leaving only the nanoparticles on the TEM grid. The
temperature was slowly raised to -55◦C using the holder’s built-in heater and
temperature controller over a period of 26 minutes, until ice was no longer vis-
ible. The temperature was then lowered again to -180◦C and the nanoparticles
re-imaged free of surrounding vitrified liquid.
To determine the effect of humidity on the shape and size of mesoporous
49
silica nanoparticles the vacuum-dried specimen was exposed for 24 hours to
open air in the lab. We then imaged the same particles again using cryo-STEM
at -180◦C. Cooling the specimen reduces ionization damage (radiolysis) due to
electron beam exposure [75].
We characterized the shape change of the particles by comparing measure-
ments of their diameters, both from corner-to-corner of the hexagon or star (dc)
and from the centers of each side to its opposite (ds). To account for uncertainty
in the measuring process, under each condition, each diameter on each particle
was measured 3 times—giving 18 measurements per particle: 3 dc each taken
3 times, and the same with ds. Each particle was then compared to itself: the
change in diameter is reported as a percentage of its diameter in the solution-
embedded state.
4.2.3 Cryo-STEM tomography
To verify the three-dimensional pore structure of the MSNs we used cryo-
tomography. The porous silica can be sensitive to high electron doses, taking on
a “spongy” appearance after beam damage, but cooling specimens slows ioniza-
tion damage during imaging and allows us to acquire a full tilt series without
damaging the particles. The STEM tilt series was collected using a Gatan 914
cryo-tomography holder. The sample was cooled only to -140◦C, balancing the
benefits of slowed damage at low temperature with deposition of amorphous
ice on the specimen when in the microscope column, which is worsened as tem-
perature decreases. At low temperature, the specimen was also likely to drift
more than room temperature imaging. To compensate for this, instead of col-
50
lecting one slow scan at each tilt angle 8 scans at 1 µs per pixel were aligned and
averaged. The tilt-series contains a total of 75 images, collected sequentially
from -74◦ to 74◦ in steps of 2◦. To increase the depth-of-field for highly tilted
specimens, we used a smaller semiconvergence angle of 4.5 mrad.
Alignment of the tilt series was computed using IMOD [76] fiducial align-
ment of gold particles. The aligned tilt series was exported, and reconstructed
using 12 iterations of simultaneous iterative reconstruction technique (SIRT) re-
construction algorithm coded in MATLAB2. Hand segmentation of the particle’s
outer shell was done using the Avizo visualization program (FEI company)3.
Visualizations shown here, as well as 3D-FFT computations, have been created
using Tomviz [77, 78].
4.3 Results
4.3.1 Cryo-STEM imaging of MSNs
To characterize the structure of MSNs in solution, cryo-STEM images were col-
lected of particles snap-frozen in 80:20 ethanol:water solution. Figure 4.2 shows
representative particles in solution, imaged in different orientations on the grid.
The first (Figure 4.2a) shows a particle oriented for imaging down the pore axis.
Increased intensity at the particle’s edges in this HAADF-STEM image indicates
a higher density of silica at the edges relative to the pore matrix: a core-shell
silica architecture. This increased density arises from the deposition of excess
2Courtesy of Dr. Robert Hovden.
3Begrudgingly.
51
Figure 4.2: Projection images of hexagonal mesoporous silica nanoparti-
cles in 80:20 ethanol:water solution in two orientations. (a) The
hexagonal shape of the particles and their arrangement of pore
structure in cross-section. Increased intensity at the edges of
the particle indicates a denser shell of silica surrounding the
pore structure. The FFT of the porous region illustrates sym-
metry in the pore structure, with pore spacing of (3.8 nm). A
second set of visible spots indicates we have resolution better
than 1.9 nm in this cryo-STEM image. (b) A particle imaged
perpendicular to the pore axis. From this orientation, it is clear
that the pores extend as columns throughout the length of the
nanoparticles.
silane species onto the particle’s surface at the conclusion of the synthesis reac-
tion [73].
In this orientation, we clearly observe the hexagonal pore arrangement of
these MSNs. The pore spacing is verified in the Fourier transform of the area in-
dicated; spots correspond to a pore spacing of 3.8 nm; higher-order spots show
resolution in this image is better than 1.9 nm. Figure 4.2b shows a particle ori-
ented such that it is lying down on the grid. In this orientation we can see that
the pores are cylindrical, extending the full length of the nanoparticles.
52
4.3.2 Imaging shape change using cryo-STEM
To image the shape change of the ss-MSNs, the same set of particles was im-
aged in solution, after dying in the microscope vacuum, and after exposure to
humid air for 24 hours (see Methods). Figure 4.3 shows a representative set
of particles as well as a higher-resolution image of one particle as it changes
shape. In solution (Figure 4.3a, b), the particles all demonstrate the same hexag-
onal shape described in the previous section. The ethanol/water solution was
next sublimated away in the microscope. After this process was complete, the
re-imaged particles still retain hexagonal shape (Figure 4.3c, d). Some conden-
sation of the silica occurs even during vacuum drying, resulting in a decrease of
the corner-to-corner diameter dc of this particle from 182 nm to 163 nm, but no
shape change has taken place in this process.
After being unloaded from the microscope and exposed to humid air for 24
hours, the particles imaged again have completely changed their shape (Figure
4.3e, f). In cross-section, the centers of each hexagonal face contracted towards
the center of the particle, resulting in a six-sided star shape. This shape change
is accompanied by a loss in clear pore structure at the edges of the particles. The
particle in Figure 4.3 decreased in diameter to 145 nm, a 20% decrease from its
size in solution. The change in shape can be quantified by comparing the size
decrease in corner-to-corner measurement, dc, with the side-to-side measure-
ment, ds, of the hexagonal cross-section. This measurement decreased 28.5%,
about 1.4 times more than the corner-to-corner measurement.
Shape changes across the specimen were characterized by analyzing images
in each of the three conditions of 66 nanoparticles, which were chosen because
they remained oriented such that they were imaged looking down the pore axis
53
Figure 4.3: Two sets of mesoporous silica particles imaged in solution (a,
b), after vacuum drying (c, d), and after exposure to humidity
(e, f). The particles’ shape changes from hexagonal to six-angle
star after the specimen was exposed to humid air in the lab for
24 hours.
in each state. Each individual particle’s shape change was measured by com-
paring dc and ds before and after the change. Figure 4.4 shows the results of
these measurements in histogram form. Firstly, we investigate the percentage
decrease in dc from solution to vacuum-dried (Figure 4.4a, c) and solution to
humidity-exposed (Figure 4.4b, d). By comparing these decreases, we can de-
scribe the shape change as a ratio of the ds contraction to that in dc. A few neg-
ative values are present. These do not indicate a growth of the particle, rather a
limitation of the method used here that small orientational changes of the par-
ticles can take place between imaging conditions to change particles’ apparent
diameters.
In the shrinkage distributions shown in Figure 4.4, the relationship between
54
Figure 4.4: Shape change across 66 particles. Overall, from solution to vac-
uum, particle size dc decreased an average of 5.0% and ds 3.9%.
After humid exposure, the total reductions were 18.9% for dc
and 13.1% for ds. The dotted lines on each distribution show
the mean; solid lines are the mean ± the standard error of the
mean.
dc and ds describes the extent of the shape change from hexagon to six-angle
star. In the vacuum dry state, we observe that the average decrease in dc is 5.0%
and in ds is 3.9%. With these similar decreases we see very little change from the
hexagonal shape observed in solution. In contrast, for the humidity-exposed
particles, ds shrinks 1.4 times the amount of dc, 18.9% compared to 13.1%.
A separate set of particles that were oriented on their long sides in the pro-
jection STEM images were measured in cryo-, vacuum-dried, and humidity-
exposed state to determine any change in the length of the particles that may
occur in addition to the cross-sectional shape change. An image of one of these
particles in cryo-state is shown in Figure 4.5a. We found that the overall change
in this length is small relative to the cross-sectional decrease in both dc and ds,
on average just above 2%. The percent decreases in length for each particle mea-
55
Figure 4.5: Decrease in particle length observed from cryo-STEM to
humidity-exposed state. (a) Cryo-STEM image of one ss-MSN,
showing orientation of particles whose lengths were measured.
(b) Histogram showing the distribution of length decreases
measured from cryo-STEM to humidity-exposed state. The av-
erage shrinkage is just over 2%.
sured are shown in Figure 4.5b. Because the length decrease is small, we expect
that the shape change is not a result of condensation of the silica matrix—in this
case we should see a length decrease similar to the cross-sectional diameters.
However, a shrinking of the pore diameters would result in the size changes
we observe here. The dense outer silica shell likely prevents the particles from
shrinking symmetrically—where the shell is weaker at the centers of its sides it
buckles inwards, leading to the six-pointed star shape.
56
Figure 4.6: MSNs synthesized in ethanol (left) and those synthesized in
water (right). At left, we observe hexagonal particles with clear
pore structure. The water-synthesized particles at right ap-
pear the same as those that have undergone humidity-induced
shape change—star-shaped and with less clear pore structure.
This is in agreement with shape change being caused by the
presence of water molecules.
4.3.3 Shape change directed by solvent used for synthesis
To verify that the shape change is indeed caused by the presence of humidity
or water and not some other nuance of the experimental process, MSNs syn-
thesized in pure ethanol were compared to those synthesized in pure water
by imaging with cryo-TEM. Figure 4.6 shows the results. Particles in ethanol
display the symmetric hexagonal shape with uniform, columnar pores. Parti-
cles synthesized in water conversely are star-shaped—still with sixfold symme-
try—showing the same behavior as those exposed to humidity in the shape-
change experiment above.
57
4.3.4 Three-dimensional structure of MSNs
To better understand the three-dimensional architecture of the ss-MSNs, we
turn to STEM tomography. The low temperatures used in cryo-STEM reduced
beam damage sufficiently to allow collection of a tomographic tilt series. Con-
ventional room temperature TEM and STEM cause shifting and morphing of
the particle structure due to beam damage which can be mitigated by careful
image collection techniques at low temperature.
The tomographic reconstruction is shown in Figure 4.7. The reconstruction
verifies the pore structure in three dimensions, with hexagonally arranged pores
(Figure 4.7a) that extend the length of the nanoparticle (Figure 4.7b). Both Fig-
ure 4.7a and b show sums through the tomographic reconstruction; 50 nm in a
plane perpendicular to the pore axis in Figure 4.7a and 25 nm for a plane slicing
through the pores in Figure 4.7b.
In the reconstruction, we also observe the denser outer silica shell, which is
apparent in the views in Figure 4.7a, b, and which we hand segmented in Figure
4.7c.
Visualization of the reconstruction in three-dimensional Fourier space re-
veals the nanoparticles’ pore spacing, arrangement in columns, and the location
of each individual tilt projection image (Figure 4.7d-f). When calculating the
three-dimensional Fourier transform, we used the periodic plus smooth decom-
position to remove artifacts due to discontinuities in intensity at the edges of the
reconstruction [79]. The orientation of the spots in three dimensions reveals the
pore geometry: hexagonally arranged spots viewed along the pore axis (Fig-
ure 4.7d) show the pore spacing; rotated 90◦ these spots lie in one plane only,
58
Figure 4.7: Cryo-STEM tomography reconstruction of a vacuum-dried
mesoporous silica nanoparticle. Slices through the recon-
structed volume show the pore structure perpendicular to (a)
and along the pore axis. (b) Locations of the slicing volumes
and hand-segmented outer shape of the particle are shown
in (c). The three-dimensional FFT clarifies the pore structure.
Looking down the pore axis, spots in the FFT illustrate the
hexagonal pore arrangement (d). From the side, these spots
are arranged in one plane (e). The 3D FFT when viewed along
the tilt axis clearly shows the locations of the missing wedge
and each of the tilt projection images (f).
59
confirming that pores are columns persisting through the depth of the particle,
rather than a three-dimensional honeycomb shape. Viewing the 3D FFT along
the tilt axis (Figure 4.7f) reveals the angle of each tilt projection and the missing
wedge which here is oriented vertically.
4.4 Conclusions
We used cryo-STEM to observe mesoporous silica nanoparticles in solution, af-
ter vacuum drying, and upon exposure to humid air and have demonstrated
this technique’s utility for imaging soft materials samples both due to its im-
proved contrast over cryo-TEM for heavier, inorganic material samples as well
as the decrease in radiolysis that occurs at low temperature. We observed that
the presence of water causes nanoparticle shape to change from hexagonal to
six-angle star. Cryo-STEM enabled measurements of the particles in each condi-
tion to quantitatively describe the shape change: on average particles decreased
13.1% in diameter from corner-to-corner and 18.9%, 1.4 times more, in side-to-
side diameter after exposure to humid air. This phenomenon suggests a need
for imaging specimens in solution to avoid possible structural changes arising
from conventional sample preparation.
60
CHAPTER 5
BIOLOGICAL CRYO-STEM IMAGING USING THE ELECTRON
MICROSCOPE PIXEL ARRAY DETECTOR
The EMPAD is a direct electron detector optimized for STEM which col-
lects the full CBED pattern at each scan pixel of an image. The result is a
four-dimensional dataset which contains all the electrons incident on the spec-
imen. We use this newly-developed detector to image frozen-hydrated biolog-
ical specimens: both thick E. coli cells (up to 1 µm) as well as thin, purified
ribosomes. Compared to conventional STEM detectors, the EMPAD provides
improved performance for radiation sensitive specimens because it collects all
incident electrons. We discuss practicalities of STEM imaging with the detec-
tor, and show equivalents to conventional BF and ADF images. The pixelated
detection of the full 2D CBED pattern at each scan position using the EMPAD
also allows new types of imaging. Here, we discuss tilt-corrected bright field
STEM (tcBF-STEM), comparing it to results from TEM and zero-loss EFTEM.
We show marked improvements in image quality from thick regions of cellular
specimens especially at very low electron dose. We also explore possibilities for
future experiments with the detector, including improvements to tcBF-STEM,
tomography, and ptychography in biological specimens.
5.1 Introduction
Cryo-transmission electron microscopy (cryo-TEM) has enabled new insights
into biology as it allows the study of specimens preserved in near-native state.
Single-particle reconstructions, propelled by advances in direct-detector tech-
nology, have reached near-atomic resolution [80, 52]. At the same time, ad-
61
vances in cryo-electron tomography provide ever-smaller details of complex 3D
structures of larger systems, including whole cells [81].
STEM, however, has been rarely used in biological cryo-EM studies (recall
Chapter 1.2.6). In this chapter, we discuss new possibilities for cryo-STEM imag-
ing of biological specimens using a new type of STEM detector. As TEM imag-
ing has experienced a resolution boost from direct detection, now STEM imag-
ing is undergoing its own detection revolution: 4D STEM in which the full con-
vergent beam electron diffraction (CBED) pattern is captured at each scan pixel
[82]. This has already enabled new types of imaging in materials science includ-
ing post-acquisition dark-field STEM crystal mapping, lattice constant mapping
[83], true center-of-mass mapping enabling the measurement of magnetic fields
from deflections in the diffraction pattern [84], and full-field ptychographic re-
construction of the atomic potential in 2D materials to record-breaking resolu-
tion [85]. We describe the first application of such a detection scheme to biolog-
ical cryo-STEM using the electron microscope pixel array detector (EMPAD) at
Cornell.
5.2 The electron microscope pixel array detector
For the most complete description of the detector and its properties, see refer-
ence [84]. Here, we focus mostly on the properties of the EMPAD which im-
prove its utility in the field of biology and for cryo-STEM experiments.
The EMPAD is a direct electron detector optimized for STEM [84]. Its sensor
comprises 128x128 150-um pixels individually bump-bonded to readout elec-
tronics. With a dynamic range of 1,000,000 to 1 it detects the forward-scattered
62
Figure 5.1: Images of the EMPAD. a The detector, mounted on the micro-
scope, sits in the same plane as the HAADF detector. (b), (c)
Images of the insert and detector itself with housing removed.
Figure from [84].
beam without saturating as well as single electrons scattered to higher angles.
Minimizing the effect of specimen drift, especially prevalent at low temperature,
requires a fast readout speed. The EMPAD has a readout time of 0.86 ms/pixel,
important for reducing drift especially at low temperatures. With integration
time of 1 ms used here, the total time for one frame of 256x256 pixels is 2 min-
utes. At low doses required to image frozen-hydrated cells, a high signal-to-
noise ratio (SNR) is necessary. Measured at 300 kV, the SNR for a single electron
is 2071.
The detector mounted on the microscope is shown in Figure 5.1 [84].
The EMPAD replaces STEM detectors in the diffraction plane, but otherwise
operation of the STEM is unchanged. Figure 5.2 illustrates the detector place-
ment, positioned to collect the full scattered beam at each scan position. In
contrast to the overlaid conventional ADF and BF detectors the EMPAD has the
ability to collect the full transmitted CBED pattern at each scan pixel. This al-
1By the method described in Tate et al. [84], the single-electron peak is at 579 units, with
width of the zero-electron peak of 2.8; the ratio gives 207.
63
Figure 5.2: Coherent detection using the EMPAD. (a) A pixelated detector
replaces conventional HAADF (orange) and BF (red) detectors
in the diffraction plane to collect the entire CBED pattern. (b)
Typically, small detectors relative to the size of the BF disk are
used to produce a coherent image. If the entire BF disk is used,
an incoherent image results. (c) EMPAD pixels located within
the bright field disk are each a coherent detector though lo-
cated off the optical axis. (d) Off-axis STEM is equivalent by
reciprocity to tilted-beam TEM (e), leading to visible shifts in
images formed by off-axis BF STEM detectors.
lows information to be extracted from any electron incident on the sample to
form an image.
Figure 5.3a shows one CBED pattern on the detector. The intensity of each
pixel can be directly interpreted as electron counts. Here, the total number of
electrons in the CBED is 4578 and the scan pixel size 3.9nm corresponding to
a dose of 3 e−/Å2. If the CBEDs are magnified by changing the camera length,
electrons at higher angles are not collected and summing is no longer an ac-
curate way to determine the electron dose on the specimen. In these cases a
reference collected with the same acquisition conditions over vacuum gives the
64
exact dose rate of the incident beam.
5.3 Imaging biological specimens with the EMPAD
5.3.1 Classes of biological specimens
In the discussion here, we address mainly two types of specimens common in
biology. The first is a purified protein or macromolecule plunge-frozen in so-
lution—a candidate for single particle analysis in cryo-TEM. By biological stan-
dards this is a thin specimen—approximately 70 –100 nm in thickness at the
center of a grid hole [86]. In this thesis, we imaged purified ribosomes (the 80S
S. cerevisiae ribosome) as a thin test specimen.
At the other end of the thickness spectrum, we have specimens more than
a few hundred nanometers thick, consisting of whole cells and thick sections.
Here, we use whole plunge-frozen E. coli cells, which are approximately 800 nm
in thickness.
Thus far, thick specimens have seen the greatest benefit from imaging in the
STEM. This is due to the lessened effect of chromatic aberration in the STEM
optical system. In thick specimens, electrons are far more likely to scatter in-
elastically. The inelastic mean free path of an electron in vitreous ice is only
around 300 nm, so for specimens close to a micron, such as the interior regions
of the E coli cells, most electrons will have lost energy as they travel through
the specimen. Recall from Chapter 1 that in STEM imaging the incident elec-
tron beam is focused to a probe and scanned across the specimen. This is in
65
Figure 5.3: STEM imaging of E. coli using the EMPAD. (a) Convergent
beam electron diffraction pattern from one scan pixel. The
entire transmitted beam, resolved by scattering angle, can be
collected at each scan position. This creates flexibility in con-
trolling the signal contributing to the final image. The number
of electrons incident at any pixel can be read off directly with
single-electron events evident at high angles. Overlays show
locations of conventional STEM detectors. (b) Annular dark
field image formed by integrating detector pixels in an annulus
located between 3x the probe convergence angle and the edges
of the detector (orange overlay in (a)). (c) Bright field image
formed by integrating up to 1/3 the probe convergence angle
(red overlay in (a)). (d) Images formed by more complicated
methods than simple integration—here the image intensity at
each pixel is equal to the second moment of its CBED pattern.
66
contrast to TEM imaging, in which a wide beam passes through the specimen
and is subsequently focused by the objective lens. Because the beam is focused
after electrons have lost energy in the specimen, the range of energies in the
beam leads to chromatic blurring. One way to remove this effect is to insert a
post-specimen energy filter and keep only those electrons which have not lost
energy in the image but this greatly reduces the fraction of incident dose used
to form the image which can mean a large loss of resolution in dose-sensitive
frozen-hydrated specimens. In STEM electrons still lose energy in the specimen
but because all the imaging optics are located above the specimen there is less
chromatic effect.
In thick specimens one also has to account for multiple scattering, which
in the STEM translates to beam broadening. Multiply-scattered electrons tend
to scatter to high angles, thus this effect can be limited by using axial, or bright
field detection [48]. Similar to this effect is geometrical beam divergence in thick
specimens. Here, a beam focused at the top surface of the specimen is spread
at the bottom surface, simply due to the geometry of the focused probe. Reduc-
ing the convergence angle of the probe reduces this beam divergence and also
increases the depth of field [87]. In the experiments here, we use a semiconver-
gence angle α of 2 mrad for the cellular specimens. This gives us a depth of field
(DOF) of 363 nm as shown in Equation 5.1. For comparison, with an aberration-
corrected probe with a semiconvergence angle of 30 mrad, depth of field is only
1.6 nm. This creates a severe depth sectioning effect, where only a small fraction
of the sample’s height is imaged in focus.
0.74λ
DOF = (5.1)
sin2 α
67
The tradeoff of using a small convergence angle is an increase in probe size,
which is calculated using Equation 5.2.
0.61λ
d = (5.2)
sinα
For a 2 mrad probe at focus, this probe size is 6 Å. This value puts a fun-
damental limit on the resolution of scanned imaging, however it remains well
below the size of features imaged at the cellular level.
Imaging with the EMPAD
To replicate images formed with a traditional STEM detector setup the relevant
area is defined in the detector image/CBED space as in Figure 1a and Figure 2a.
By summing the intensity within each of these regions we can construct a bright-
field image (red region, Figure 2b) or annular dark field (orange region, figure
2c) image. The pixelation of the detector allows complete freedom in choosing
regions to form images such as inner and outer angles of an ADF image.
More complicated imaging methods than simple integration become pos-
sible with pixelated detection of the CBED pattern. One example is second-
moment imaging, shown in Figure 2d. The intensity at each pixel is the second-
moment of its CBED—the intensity weighted according to the square of its
distance from the center. Similar in appearance to an ADF image, the second-
moment image shows the scattering power of the specimen[88]. Since the low-
Z-number components of biological specimens tend to not scatter highly to the
usual HAADF region this type of dark-field imaging may prove more useful,
and certainly more convenient because the lower-scattering information is au-
68
tomatically retained.
Unlike imaging with traditional detectors, we are not simply presented with
one image after the scan is complete. The data is saved in raw block format
as 32-bit floats. After reading the data into the computer careful analysis is
needed to make even the simplest integrated images—for example, accurately
determining the center of the CBED patterns to properly place a bright- or dark-
field detector. A set of Python codes has been developed for the data analysis;
the code, instructions, and descriptions are presented in Appendix B.
5.4 Tilt-corrected bright field STEM imaging
For thick specimens, such as the E. coli cells imaged here, axial detection (bright-
field imaging) provides higher-resolution images than dark-field. Dark-field
imaging detects more multiply-scattered electrons displaced from the incident
beam and scattered to higher angles, while bright-field is less susceptible to this
beam-broadening [87, 48]. Additionally, for light elements present in biolog-
ical specimens phase contrast in bright field can provide better contrast than
Z-contrast in dark-field imaging, and additional contrast can be generated by
defocusing, similar to TEM.
To record a phase-contrast image a coherent detector must be used. With
a conventional bright-field detector, this can only be achieved by using a small
detector relative to the convergence angle of the probe (α); typically the detector
size is limited to 1/3α (Figure 5.2b). This severely limits the number of incident
electrons that can be used to form the image, leading to reduced resolution for
dose-sensitive specimens. Incoherent bright-field imaging is also possible when
69
imaging in focus—a detector close to the size of the full bright-field disk pro-
vides the complement of the incoherent ADF image [89].
The EMPAD allows the detection of coherent images at any point in the
bright field disk. Each pixel of the detector can be thought of as one coherent
bright field detector—it subtends an angle much smaller than the convergence
angle of the beam. Most of these detectors are located off the optical axis, and
can be thought of as being formed with a beam tilted relative to the axis. The
effect of imaging with an off-axis detector is equivalent to imaging in TEM with
a tilted beam (Figure 5.2d). This leads to a shift in the image related to the aber-
rations present, primarily defocus [90]. The amounts of the shifts are given by
the derivative of the aberration function [91, 92, 93].
These shifts are illustrated for a defocused image (4 µm) in Figure 5.4. A
conventional, coherent bright field image formed with a small on-axis detec-
tor is shown in Figure 5.4a. If the detector is expanded to the full bright-field
disk to increase signal, the resulting image is blurred due to the image shifts
from higher tilt contributions (Figure 5.4b). The effect is more clearly illustrated
in Figure 5.4c, where images formed from a few bright field pixels at the edge
of the bright field disk, opposite the optical axis from each other, are superim-
posed. A shift of a few pixels at the E. coli membrane is discernible by eye.
Fortunately, we can correct for these shifts pixel-by-pixel. To do this, an
image is formed from each pixel in the bright field disk (Figure 5.4d). A shift
can be precisely calculated for each image using cross-correlation. Specifically,
we cross-correlate each image with every other image in the bright field disk
and average the positions of the cross-correlation peak to determine the correct
offset for each image. The resulting shift map is shown in Figure 5.4e, top. This
70
Figure 5.4: Method of tilt correction for efficient coherent bright field
imaging. (a) An integrated image formed using electrons scat-
tered within 1/3 of the probe convergence angle, representing
a typical image formed with a conventional bright field detec-
tor. (b) Image formed using the majority of electrons in the
bright field disk. The image has higher signal than in (a), but is
blurred due to the effect of defocus aberration. (c) Images from
small-off axis detectors (3x3 pixels) show shifts relative to each
other—the source of the blurring in (b). The relative shift can be
determined by cross-correlation. (d) An image formed by only
one BF pixel. These individual images are cross-correlated to
produce a shift map and final aligned image shown in (e).
71
shift map is exactly a map of the aberrations in the probe and fitting the shifts
to the aberration function has been demonstrated as a method for diagnosing
aberrations in the STEM [93]. Once the shifts for each pixel are known, they are
applied to the individual images which are then summed to produce the final
tilt-corrected bright field image (Figure 5.4e, top).
The tilt-correction method also can provide an increase in real-space resolu-
tion of the final image compared to the sampling in the recorded 4D dataset. Re-
dundant information in diffraction space is used to fill in information between
real-space pixels in the scan. When the probe is defocused, a shadow image is
formed in the BF disk of the diffraction pattern with better sampling than the
real-space scan. This information is coded in the shifts determined between the
images formed from each bright field pixel, which are determined to sub pixel
resolution by fitting the cross-correlation peak. Figure 5.4(e) has been upsam-
pled by 4 immediately prior to applying the shifts. The upsampled tilt-corrected
image shows smoother detail by filling in information between the scan pixels.
Acquisition parameters can be optimized in a few ways to create the best
tilt-corrected image. Firstly, in order for upsampling to properly fill informa-
tion between pixels the defocus should be optimized so that the shifts in indi-
vidual images span a range of at least ±1/2 pixel. The only benefit of increasing
defocus beyond this point will be to increase contrast in the final image, how-
ever this can come at the expense of high-resolution information. Secondly,
each individual bright field image can be made more coherent by magnifying
the CBED pattern (increasing the camera length) so that each pixel subtends a
smaller angle in diffraction space. This must be balanced with the requirement
that each individual pixel contain sufficient signal to successfully determine the
72
shift through cross-correlation.
5.4.1 The contrast transfer function in tcBF-STEM
The CTF for axial bright-field STEM is intuitively the same as that for conven-
tional TEM due to the principle of reciprocity. However, for the tilt-corrected
method, we have the added complexity that each image is located at a different
tilt angle from the optical axis or in other words collected with an off-axis BF
detector.
The phase CTF for tilted illumination, or for an off-axis STEM detector, is
shown in Equation 5.3, from [94, 95].
{ [ ]
t̃p (k) = i exp[ −i (χ (k0 + k) − χ (k0])) × E (k0 + k,k0) (5.3)− exp i (χ (k0 − k) − χ (k0)) × E (k0 − }k,k)
Here, k is the spatial frequency, k0 is the spatial frequency of the detector, or
here, the location in CBED/frequency space of the relevant pixel from which we
take our bright field image. χ is the aberration function (which we’ve seen can
be determined from the CBED shift map [93]). E is the envelope function de-
scribing the falloff of the CTF at high spatial frequency due to beam divergence
and spread in defocus. We ignore this function for the rest of the discussion, as
the most interesting behavior is the frequency-dependent behavior, especially
the location of zero-crossings of the CTF.
For an axial detector, k0 = 0 and the equation simplifies to Equation 5.4.
73
[ ]
t̃ (k) = 2 sin χ (k) E (0,k) (5.4)
This is the familiar equation for both TEM and BF-STEM imaging.
The amplitude CTF for a tilted beam is similar in form to the phase CTF.
Here we consider the amplitude CTF in addition to phase to better understand
the contrast transfer in the thick E. coli specimens, where we expect far more
amplitude effects than thin specimens. The amplitude CTF is given in Equation
5.5 [95].
[ ]
t̃a (k) = exp [−i (χ (k0 + k) − χ (k0)]) × E (k0 + k,k0) (5.5)
+ exp i (χ (k0 − k) − χ (k0)) × E (k0 − k,k)
Figure 5.5 shows simplified CTFs for zero beam tilt. We consider the spatial
frequencies in one dimension and use an aberration function that only includes
defocus ∆ f and spherical aberration Cs (Equation 5.6).
( )
1
χ (k) = π Csλ3k4 − λ∆ f 2k2 (5.6)2
The plot of the untilted CTF is shown in Figure 5.5 using a Cs value of 1.2
mm and a defocus of 10 nm. The plot shows the familiar, oscillating form.
Let’s now include pixels tilted off of the optical axis. The tilts were assigned
according to a real EMPAD dataset, collected with a semiconvergence angle of 2
mrad, with the radius of the central disk 15 pixels. Each pixel is then assigned a
spatial frequency (k0 in Equation 5.3). The CTFs for all 15 pixels were calculated
and are plotted in Figure 5.6.
74
Figure 5.5: The bright field phase (left) and amplitude (right) CTF for an
axial STEM detector, calculated for Cs = 1.2 mm and ∆ f = 10
nm.
Figure 5.6: Phase (left) and amplitude (right) contrast transfer function of
off-axis BF detectors: 15 CBED pixels along a line from the cen-
ter of the disk to the edge. Going to higher tilt shifts the posi-
tions of the zero crossings of the CTF.
75
In the plot we observe small changes in the CTF from pixel to pixel. Sim-
ilar to increasing the defocus, increasing the detector’s displacement from the
optical axis brings the position of the first zero crossing closer to zero. This has
some interesting implications: the information that we usually consider “miss-
ing” at the spatial frequencies where the zero crossings occur is actually present
in some of the other tilts. More generally, we really have n images with n slightly
varying contrast transfer functions.
Currently, and in all the results presented here, we combine the images by
simple addition. Given what we know about the CTF now this seems somewhat
naive; however, getting to the different information at each tilt takes some work.
For the thin class of specimens where we have little amplitude contrast, we may
be able to correct for the CTF for each tilt similar to single particle analysis, in
which large TEM datasets are collected at a few different defocus values in or-
der to fill in gaps in the CTF [96]. This would entail very accurate determination
of the aberration coefficients from the tilt map as in [93]. The CTF (in two di-
mensions) for each pixel can then be calculated from the known pixel tilts, and
the image corrected in Fourier space. Attempts at this have so far been unsuc-
cessful, and for thick specimens such as the cells shown here correction would
not be straightforward.
76
5.5 Results
5.5.1 Comparison of tcBF-STEM, conventional TEM, and zero-
loss EFTEM for imaging E. coli
To compare tcBF-STEM with commonly used TEM and zero-loss EFTEM, we
imaged the same area of one E. coli cell at the same dose using all three meth-
ods. Figure 5.7 shows images collected using each technique at a dose around
7.5 e/Å2. To determine the performance of each method, we look for clear res-
olution of ribosomes in the thickest region of the cell ( 800 nm in thickness), as
well as the 4 nm spacing of the membrane bilayers at the outer edges.
A conventional TEM image of the cell (Figure 5.7a) shows features near the
thin edges of the cell, but chromatic blur makes interior features indiscernible
at more than approximately 200 nm from the membrane. To remove the effect
of chromatic aberration, we placed a 10 eV slit at the zero-loss peak. Figure
5.7b shows the resulting filtered image. Ribosomes can now be observed in the
thick region of the cell, however, much of the incident dose now is lost due to
the energy filter resulting in a large drop in intensity at the thick regions of the
cell. A direct comparison of counts on the detector with and without the energy
filter shows that the EFTEM image uses only 18% of the electrons used to form
the TEM image. This is suboptimal because the remaining electrons still pass
through and damage the sample but are discarded by the filter and unused.
The tcBF-STEM image (Figure 5.7c) shows improved signal in the cell’s inte-
rior compared to EFTEM and resolves the ribosomes. To more directly compare
the features visible in the interior across the three methods, we took Fourier
77
Figure 5.7: Images of the same cell taken using TEM, zero-loss EFTEM,
and tcBF-STEM at dose of 7.5 e/Å2. The unfiltered TEM image
shows little detail and chromatic blurring in the thick center re-
gion of the cell (left). Zero-loss-filtered TEM has less blurring in
the thick region but suffers from a loss of signal where the ma-
jority of electrons have lost energy and are removed by the en-
ergy filter. This image uses only 18% of the electrons present in
the TEM image (center). tcBF-STEM of the same region shows
resolution of details in the interior of the cell without loss of sig-
nal in this thick region (right). More magnified images below
demonstrate resolution of the 4 nm membrane bilayer spacing
most easily discernible for the tcBF-STEM image; scale bar for
these is 50 nm. FFTs were calculated from thick regions of the
cells. In the EFTEM and tcBF-STEM FFTs a ring is visible cor-
responding to the diameter of ribosomes present in this thick
region. Scale bar for FFTs is 0.1 nm−1.
78
transforms in the thick region (insets). In the EFTEM and tcBF-STEM images, a
ring is visible in the FFTs corresponding to the ribosome diameter of 28 nm.
In addition to the FFT, we used a Fourier ring correlation (FRC) to investigate
information transfer and relative resolution of the three techniques in thick and
thin regions of each image [97]. The results are shown in Figure 5.8. The FRC
measures the similarity between two images by cross-correlating correspond-
ing rings in Fourier space. The output is the correlation as a function of spa-
tial frequency. The corresponding method in three dimensions, Fourier shell
correlation, is often used as a resolution measure in three-dimensional recon-
structions of biological macromolecules in single particle analysis. To generate
the two datasets/images to correlate we used alternating bright field pixels for
the tcBF reconstruction and alternating exposures (two out of four total) for the
TEM images, which were aligned and summed. As in Figure 5.7, the TEM and
EFTEM images are resampled to match the pixel size of the tcBF-STEM image,
however for the FRC analysis we use the non-upsampled pixel size of 2.6 nm.
The images were symmetrized by reflecting over the edges when taking FFTs to
prevent intensity discontinuities and edge effects from dominating the FRC.
The FRC analysis shown in Figure 5.8 further underscores the improvements
we see in tcBF-STEM imaging over TEM and EFTEM. Most obviously, the FRC
for tcBF-STEM in both the thick and thin regions is higher than in TEM and
EFTEM at all spatial frequencies in the image. For all three imaging modes we
observe better resolution (higher FRC at high spatial frequencies) for the thin
region compared to the thick. tcBF-STEM has the smallest difference between
the FRC of the thick region and the thin, showing that the specimen thickness
has less effect on the resolution for this method.
79
Figure 5.8: Fourier ring correlations of thick and thin regions of the images
in Figure 5.7. tcBF-STEM shows the highest FRC over all spatial
frequencies for both thin and thick regions. Scale bars 100 nm.
To achieve similar contrast between the methods, the tcBF-STEM image was
collected at 2.5 µm defocus while the TEM images at 7.5 um. This may con-
tribute to clearer resolution of the membrane bilayer in the tcBF-STEM image,
indicated in the inset with an arrow, as a higher defocus will limit the resolution.
The tcBF-STEM technique will be especially useful for tomography of cel-
lular samples, allowing reconstruction of samples with higher thickness than
TEM. For this application the dose must be lowered for each acquisition so that
the collection of images across a wide tilt range remains lower than the damage
threshold of the specimen. To assess the technique’s performance at low dose,
in Figure 5.9 we compare tcBF-STEM to TEM and EFTEM at an electron dose of
0.5 e/Å2, and at defoci of 0.5 µm (top row) and 4 µm (bottom row). Despite the
80
low dose, measurement of the shifts for tilt correction is successful. tcBF-STEM
demonstrates improvements in contrast and SNR at both defoci compared to
the corresponding TEM and EFTEM images. At 4 µm defocus a ring represent-
ing the ribosome diameter is again observed in the FFT of tcBF-STEM image
from the interior of the cell. tcBF-STEM therefore may offer improved 3D recon-
struction of cells compared to EFTEM.
5.5.2 Tolerable electron dose for tcBF-STEM
To determine the improvement in images with increasing electron dose for tcBF-
STEM, we collected images in one region consecutively with increasing dose,
ranging from 1.5 e/Å2 to 210 e/Å2 in one frame. The results are pictured in Fig-
ure 5.10. As expected, image quality is improved with increase in dose. How-
ever, we observe markedly improved tolerance to high electron doses with the
scanned probe in STEM when compared to parallel-beam illumination, which
has been observed before [49, 98]. Even in the final image, after a cumulative
exposure of 280 e/Å2, there is no apparent damage to the cell which is usu-
ally observed in the form of bubbling. In TEM imaging, it has been shown that
high-resolution structural information is the first to be lost as incident electron
dose increases—much higher resolution than is possible with the cellular-scale
images shown here [99, 100]. The improved dose tolerance will provide addi-
tional benefit for electron tomography, allowing collection of more tilt angles
which fundamentally increases the achievable resolution in the 3D reconstruc-
tions [81].
81
Figure 5.9: tcBF-STEM compared to TEM and zero-loss EFTEM at a dose
of 0.5 e−/Å2. The top row of images ((a), (b), (c)) were collected
at 0.5 microns defocus and the bottom ((d), (e), (f)) at 4 microns.
TEM images (a), (d) show low contrast; at low defocus almost
no features are visible apart from the outer membrane. Zero-
loss filtered images (b), (e) give a more accurate impression of
the cell’s thickness, but do not resolve features in the interior
regions. tcBF-STEM images (c), (f) show improved detail in
the thick area of the cell compared to EFTEM at the same defo-
cus. Using tcBF-STEM at high defocus we clearly observe the
ribosome diameter in the inset FFT calculated from the central
region of the cell indicated. At each defocus, contrast in the
interior regions of the cell is highest in the tcBF-STEM image.
82
Figure 5.10: Tilt-corrected BF-STEM images collected sequentially with in-
creasing dose rates per frame: (a) 1.3 e/Å2, (b) 15.2 e/Å2, (c)
57.6 e/Å2, (d) 210.5 e/Å2. The specimen appears tolerant to
a high cumulative dose, with no bubbling appearing even in
the final exposure where the cumulative dose is 286 e−/Å2.
5.6 tcBF-STEM in thin specimens
We investigate the performance of tcBF-STEM in thin specimens as well. Be-
cause there is less multiple scattering in thin specimens, more of the dose is
contained within the bright field disk. This is shown experimentally in Figure
5.11, where the radial distribution of the electrons collected by the EMPAD is
compared for an E. coli specimen and for the ribosomes. We see that in the thin
specimens, more than 70% of the incident dose is contained within the bright
83
Figure 5.11: Scattering profiles in thin ribosome (left) and thick E. coli
(right) specimens calculated using data from the EMPAD. In
thin specimens, most electrons scatter into the bright field
disk. Because of this, a bright field method makes very ef-
ficient use of the incident dose, more so than in thick speci-
mens.
field disk, compared with less than 35% for the cells. So, while the improve-
ment in chromatic aberration should be better for thick specimens, the technique
makes better use of dose for the thin specimens.
One tcBF image of the ribosomes and a typical CBED are shown in Fig-
ure 5.12. For these specimens, we used a larger semiconvergence angle of 5
mrad because the thin ribosomes require a much smaller depth of field than the
cells—5 mrad gives a depth of field of 58 nm. At higher-angled CBED pixels,
the aberration-induced shifts are also much higher, as the shifts are linear in de-
focus. For these specimens, very little defocus is needed to generate shifts of 1-2
pixels. Figure 5.13 illustrates this effect.
In the tcBF images of thin specimens, we observe good contrast and signal
to noise ratio. In Figure 5.12 we see clear detail of the ribosome structure. Here,
84
Figure 5.12: tcBF-STEM imaging of ribosomes using the EMPAD. (a) The
tcBF-STEM image, at a dose rate of approximately 30 e−/Å2
and a defocus of approximately 100 nm.
Figure 5.13: Shifts calculated in x, y for an E. coli specimen (right) with
semiconvergence angle of 2 mrad and defocus of 1.5 µm and
ribosomes (left) with semiconvergence angle 5 mrad and de-
focus 100 nm. Use of a higher convergence angle makes ac-
cessing larger shifts possible with less defocus. CBEDs are
different magnifications because of a change in camera length
between the images.
85
the tcBF-STEM reconstruction has decreased the pixel size from 1.52 nm to 3.8 Å
through upsampling. We used 128x128 pixels in the scan to minimize the drift.
For now however imaging small particles with sufficient pixel density for
high resolution imaging on the EMPAD is difficult: for 5 Å scan pixels, we need
60 pixels scanned across the diameter of a ribosome. Taking scans of 256x256
fills 1/4 of the field with one ribosome at this scan density but scanning at
256x256 requires at least 56 seconds (considering the 0.86 ms readout time as
the minimum). This is more than enough time for the holder to drift and dis-
tort the image. Even if we can realize a resolution improvement over TEM, it
will require a faster detector to decrease the effect of drift2. Compared to TEM
where a 4k detector allows us to image a few hundred particles in a single frame
with comparable pixel size even after upsampling in STEM, the convenience of
TEM coupled with reduced drift in this case makes it a more appealing option
currently for thin specimens.
5.7 Possibilities for electron ptychography in biological speci-
mens
Because the tcBF-STEM method employs only the forward-scattered electrons,
imaging methods making additional use of dose scattered to high angles may
show improvements over the technique. Full-field electron ptychography, re-
cently demonstrated with this detector using electrons scattered to more than
4 times the beam semiconvergence angle to image 2D materials, is especially
2Or the possibility of using only a fraction of the sensor—32x32 pixels would be more than
sufficient for detecting just the bright field signal!
86
promising to make most efficient use of the incident dose [85]. Simulations
of ptychography for single-particle reconstruction suggest improved resolution
over cryo-TEM imaging [101].
Experimentally, however, it seems the thickness of biological specimens,
even the thin ones, prevents improved reconstructions by electron ptychog-
raphy. Successful electron ptychography has so far been demonstrated in ex-
tremely thin 2D materials [85] or slightly thicker materials with a repeating
atomic structure using multislice ptychography [102]. In the biological mate-
rials we have been limited by thickness which causes some amplitude contrast
even in the thinner class of samples and makes ptychographic reconstruction
difficult.
Examples of electron ptychography in the biological specimens are shown
in Figure 5.14. We attempted reconstructions using only the bright field signal
using the Wigner Distribution Deconvolution algorithm on both a thick spec-
imen—a mosquito sperm (right)—and a thin specimen, again the ribosomes
(left). In both cases ptychography does improve over simple bright field imag-
ing, however does not perform as well as the tilt-corrected method, primarily
due to the upsampling allowed in the tilt corrected method.
Improvements to ptychography in thicker specimens could eventually lead
to its application in biology, which would achieve the best possible use of inci-
dent dose for imaging.
87
Figure 5.14: Bright field ptychography compared to tcBF-STEM for thin
and thick biological specimens. The left column shows images
of ribosomes at the edge of a carbon hole in the grid. While
ptychography improves over the bright field image here, the
tilt-corrected result is the only one that clearly shows the ribo-
somes, indicated with arrows. The center and right columns
show images of a mosquito sperm. Here, the ptychography
looks comparable to the tilt-corrected reconstruction up to de-
tails close to the pixel level. However, looking at the inset im-
ages far right, we see that the tilt-corrected image shows more
fine detail because it allows for upsampling.
88
5.8 Conclusions
Here we demonstrate the utility of diffractive imaging techniques for imaging
frozen-hydrated specimens in the electron microscope. The pixelated STEM de-
tector provides the ideal output for cryo-STEM imaging as it allows the possi-
bility to detect all electrons incident on the specimen. In tcBF-STEM, we employ
pixelated detection of the bright-field disk to produce coherent images using all
the electrons in the central disk. This method shows clear improvements over
TEM and zero-loss EFTEM for imaging whole cells and other thick specimens.
The most remarkable improvements were observed at very low-dose when
replicating conditions for tomographic-tilt series acquisition. Though the low-
dose images are promising, the process of tracking, focusing, and acquiring are
all difficult to optimize in the current setup. During the experiments presented
here, exposure time was set at a constant 1 ms, plus readout time of 0.86 ms with
the result that feedback during tracking and focusing is much slower than a con-
ventional detector allowing readout times on a single microsecond scale. This
also has consequences for specimen drift over the long acquisition time. Recent
software developments with the detector allow user variation of exposure time,
effectively halving the total frame time. These improvements will enable more
ease in tomographic data acquisition.
In summary, we have demonstrated the use of a pixelated direct detector for
STEM imaging of cryo-preserved cells. The EMPAD can be retrofit to any ex-
isting STEM allowing straightforward adoption of the technique. This should
allow improved STEM imaging of thick frozen biological specimens with effi-
cient use of the incident dose.
89
CHAPTER 6
CONCLUSIONS
Cryo-EM has been exploding in popularity because it allows structural
preservation of biological and soft materials in aqueous solution, and because
of instrumental developments that have allowed imaging techniques to reach
higher than ever resolution.
In this work, we’ve pushed cryo-EM beyond its conventional limits to study
soft materials specimens immobilized in solution and to study frozen-hydrated
biological specimens using cryo-STEM techniques and new detector technology.
In Chapters 2, 3, 4 we studied mesoporous silica nanoparticles in solution
using cryo-EM. In Chapters 2 and 3 the formation process of two types of par-
ticles was carefully studied. Using cryo-TEM, small organic components of the
particles were visualized intact for the first time. This direct observation al-
lowed us to pinpoint synthesis parameters leading to different crystallinity in
pore structures of the particles (Chapter 2). In Chapter 3, we observed forma-
tion of single-pore nanoparticles as components for higher-ordered structures.
These particles have served as test systems but also have important applications
in fields of nanomedicine, energy storage, and catalysis.
These studies have led to recognition that in order to discern true solution-
directed dynamics in these formation processes cryo-EM should be used as
well as three-dimensional investigation of nanoparticle structures using single-
particle analysis.
The work in Chapter 4 goes beyond formation to study properties of a shape-
shifting mesoporous silica nanoparticle. In this study, we demonstrated cryo-
90
STEM of particle immobilized in the reaction solution. Because of their rela-
tively larger size and the elemental contrast between the silica and surrounding
ice, ADF STEM proved well-suited for this work. After verifying hexagonal
structure of the post-synthesis particles in solution, we investigated the effects
of drying processes on the particles. Drying in vacuum caused a small, isotropic
shrinkage in the hexagonal structure, while exposure to humid air, equivalent
to drying in the lab for TEM analysis, caused the particles to deform to six-
angle star. This work demonstrated the potential for cryo-STEM analysis under
specific conditions, as well as the importance of specimen preparation and the
potential for drying artifacts with these materials.
In Chapter 5 we study dose-efficient STEM imaging of biological specimens,
work made possible by the invention of the EMPAD. STEM imaging that al-
lows use of all the incident electrons is necessary for imaging dose-sensitive
specimens, epitomized by frozen-hydrated cells and molecules. The myriad ad-
vantages of STEM imaging are now available for a new class of specimen, and
new STEM techniques enabled by the 4D dataset will complement biological
specimens. I expect that the tcBF-STEM technique described here is the first of
many leading to the ultimate dose-efficient imaging technique of full-field pty-
chography for thin biological specimens, and other methods for extremely thick
samples.
Here, we demonstrated tcBF-STEM’s improved performance over TEM
and zero-loss EFTEM in thick specimens, especially at very low doses (1
e−/Å2)—such as are relevant for a projection image at a single tilt in electron
tomography. New software improvements with the detector such as variable
exposure time and easy delineation of separate ”search” and ”acquire” settings
91
make tomography using tcBF-STEM a good candidate for future study.
92
APPENDIX A
PRACTICAL ASPECTS OF LOW DOSE IMAGING WITH THE EMPAD
This appendix contains some notes for cryo-STEM imaging with the EM-
PAD, based on my experience using the original detector mounted on the Titan.
For the experiment I’ve done, and any experiment using a small convergence
angle, there isn’t much of a reason to use a probe-corrected tool.
Lots of aspects of the experiment may be counterintuitive to high-resolution
STEM users, the “low-dose” aspect isn’t as intuitive or as automated as TEM,
and there are plenty of small details I’ve worked out that I hope will be useful
to anyone attempting a cryo-STEM experiment!
A.1 Choice of convergence angle
Choosing the optimum convergence angle is crucial: for thick specimens we
need an increased depth of field compared to typical specimens for STEM. Ad-
ditional considerations are desired probe overlap—with a small convergence
angle overlap is typical unless scanning over very large fields of view, but for a
ptychography experiment when more than a third of the probe should overlap
each scan position, convergence angle/probe size, defocus, and scan pixel size
should be carefully considered.
To access semiconvergence angles around 2 mrad on the Titan we utilize Mi-
croprobe STEM mode. Using the 50 mrad C2 aperture and the three-condenser
system we can set the angle to this value. We have found that we can’t trust
the reading on the microscope at these settings: for example, I have had the mi-
croscope indicate the semiconvergence angle was 1.0, 1.1, or 1.4 mrad when I
93
measure it to be around 2 mrad. Always measure the convergence angle using
the tuning grid.
When working at high defocus also remember that the probe can be much
bigger. One micron of defocus for a 2 mrad probe will create a probe 4 nm larger
in diameter than the focused probe. However, it’s not completely straightfor-
ward because defocusing has essentially moved your focused probe above the
specimen, and the probe diameter will vary throughout the thickness of the
specimen as well. Remember from Hyun et. al. the difference between focusing
at the top vs. the bottom of the specimen [87]. The choice of small convergence
angle reduces this effect.
A.2 Alignment
Setting up STEM with a small convergence angle is quite straightforward, as
there aren’t many aberrations to correct for in such a small disk. One useful
check is the beam tilt pivot points, available in “Direct Alignments” when imag-
ing the probe (clicked out of Diffraction). A good alignment here will minimize
shifts of the center of the diffraction pattern as the beam is scanned. This effect
can be corrected post-acquisition (see Appendix B), but it will save time and
energy to align well here.
94
A.3 Operating at low dose
A.3.1 Setting and measuring dose rate
The dose is set using the monochromator. This allows relative simplicity in
modulating the dose for different magnifications, or for different conditions for
searching, focusing, and exposing (certainly compared to using the spot size).
Measuring the incident dose has been done using the EMPAD itself. A small
scan taken over vacuum and averaged gives the dose per scan pixel. Dose rate
is then calibrated using the exposure time, acquisition pixel density, and field
of view. My approach to measuring the field of view has been to match areas
imaged with the EMPAD and with TEM, or to successively line up EMPAD
scans of the gold tuning grid, where the gold Bragg spots in the FFT calibrate
images at high magnification, and at lower magnifications finding the field of
view relative to the higher magnification images by matching the scan areas in
real space.
A simple Python script that can be run on the EMPAD computer during
imaging to measure the dose is below.
1 import numpy as np
2 from os import listdir, getcwd
3
4 def readPAD(filename):
5 scanx = int(filename.rstrip('.raw').split('_')[-1].lstrip('abcdefghijklmnopqrstuvwxyz'))
6 #print scanx
7 scany = int(filename.rstrip('.raw').split('_')[-2].lstrip('abcdefghijklmnopqrstuvwxyz'))
8 #print scany
9 contents = np.fromfile(filename, dtype='float32')
10 return np.reshape(contents, (128,130,scanx, scany), order='F')
11
12 number = raw_input('which file')
13
95
14 filelist = [f for f in listdir(getcwd()) if f.endswith('.raw') and number in f.split('_')]
15 for file in filelist:
16 #print file
17 data = readPAD(file)[1:-2, 3:-2, :, :]
18 electrons = np.sum(data, axis=None)/600
19 print "File {} has electrons = {}".format(file.split('_')[0],electrons)
A.3.2 Focusing and finding your specimen
Focusing can be close to impossible for biological specimens in the PAD, due
mostly to the fact that everything damages before focusing can be done well.
This, coupled with the slow scan speed (it was 1 ms only when this data was
taken) meant in order to have fast enough feedback while focusing, only a very
small scan area could be used. With the faster scans now available since the
software’s been updated this may be simpler.
The easiest way to focus using the EMPAD is to add some heavy particles to
the specimen. We typically used 5 nm gold nanoparticles. By incubating them
on the grid before plunging (2.5 µL of particles, let fully dry) we guaranteed a
good coverage of the carbon film with gold. Zooming up to a carbon area and
viewing using the HAADF mask is usually enough to find gold to focus on.
Finding the specimen can also be difficult, mostly in the case of small
molecules. The E. coli, thick and a few microns long, are reasonably easy to find
when viewing at low magnification. However, the ribosome specimens were
virtually invisible. Even after an acquisition, they were not always apparent in
the resulting image, often difficult to distinguish from ice contamination. After
bright field reconstruction they are more easily identified, but this requires a
large time investment before being able to tell if the data is good. Like any ex-
96
periment it’s best to spend time optimizing the specimen: pre-screen with TEM,
make sure there is a good density of particles on the grid. If you’re reasonably
sure that there are particles in every hole you can confidently take data knowing
you’ll likely be getting images of value.
A.3.3 Defining areas for focus and acquisition
In TEM, “Low Dose” imaging is a system in which a predefined focus area,
offset from the area of interest by a beam shift, is used for focusing at high dose
so that the area of interest is not damaged before an image is taken. In the
data presented here, we did not use a method this simple. Due to difficulty
in focusing using the EMPAD (more details below), focusing was usually done
once per grid square, as nearby as possible to the areas of interest. Focusing once
per acquisition was not done, as any defocus is compensated for in tcBF-STEM.
Updates to the software since this data was taken do permit a convenient fo-
cus area to be set up. The software has been thoughtfully designed so it presents
to the user a full scan area whose size depends on the microscope magnification.
The user can define a sub-area over which to scan. The updated software allows
two presets, ”live” and ”acquire” such that ”live” could be used for focusing in
one sub-area of the full scan, and ”acquire” for the region of interest, allowing
focusing and acquisition to be completed without moving the stage.
The ability to take many sub-scans over the full field of view without moving
the stage should also increase stability. One could imagine moving the stage to
an area of interest then taking an overview scan at low magnification. Consider
a specimen for single particle analysis, where scans should be taken in each
97
hole. The overview map would show the positions of the holes. Focusing would
be done once on the carbon film, then an acquisition scan in each of the holes
could be taken without moving the stage. Practically, I expect four holes could
be imaged without moving the stage, possibly with multiple scans in each hole.
A.3.4 Choosing an exposure time
In the data presented in this thesis, the exposure time was set at 1 ms and was
not changeable in the software. With this time, plus the 0.86 ms readout time
per pixel, a 256x256 scan takes 122 seconds, and a 128x128 takes 30 seconds. The
changeable exposure time can make a big difference: if set to 0.1 ms, a 256x256
scan takes just 32 seconds. Faster scans decrease the amount of specimen drift
over the frame.
Because of the 0.86 ms readout time, a hard minimum exists for any scan.
Thus, decreasing the number of pixels always has a greater reduction in frame
time compared to a lower exposure time.
A.4 Considerations for tomography
Like above, the biggest challenge for tcBF-STEM tomography so far has been
tracking: aligning the area of interest in the field of view, without using an elec-
tron dose high enough to damage the specimen. Our approach so far has been to
lower the magnification, greatly lower the current by increasing the monochro-
mator focus, and use a sparse scan area. Obviously, this isn’t ideal as we are
still irradiating the area of interest, and early attempts at tilt series have shown
98
damage to the specimen. Plus, tracking by this method has not been very pre-
cise. Attempts at tomography of a mosquito sperm are illustrative: the long cell
extends beyond the top and bottom of the field of view; centering the sperm
horizontally was simple, but determining where along its length the area of in-
terest was located was a challenge. The dataset remaining had only a small
section located in all the tilt images.
The new ”live” and ”acquire” settings should also make tracking and fo-
cusing easier and more convenient. Tilt series have so far been taken without
focusing between any tilts and relying on the tcBF-STEM method to account for
any induced focus changes. However, defining a focus area along the tilt axis
would allow focusing during the series, and improve the final result as each
projection would be close to the same apparent defocus. This area could also be
used to improve tracking.
New reduced exposure times should also improve tomography data, low-
ering specimen drift. One test tilt series of an E. coli specimen had obvious
differences in drift between the frames and failed to reconstruct well.
99
APPENDIX B
PAD DATA ANALYSIS USING PYTHON
As microscopists, we’re used to looking at images. The 4D dataset goes far
beyond just an image. In its simplest form it’s a stack of CBED patterns that,
while impressive, usually requires manipulation to give information about the
specimen.
My code initially was based on MATLAB code provided to me by Kayla
Nguyen and consisted of scripts for opening files, producing summed images
(BF/ADF), first- and second- moment imaging, principal components analysis,
DPC, and basic code for cross-correlating bright field images for tcBF-STEM.
The code here builds on these functions, with many additions; the most impor-
tant for the work here being preprocessing steps and improved registration for
low-dose data in collaboration with Ben Savitzky [30].
The usual steps for analysis of a data file are: reading in the data, performing
some pre-processing steps, accurately finding the center of the CBED patterns,
and forming images. The following code will perform all these steps, optimized
for biological data.
1 """
2 PAD.py: Module for reading, manipulation, analysis of 4D-STEM data from
3 the EMPAD.
4
5 This version: September 2018, Katherine Spoth
6 """
7
8 from __future__ import division, print_function
9
10 import numpy as np
11 from scipy.optimize import minimize, curve_fit
12 from scipy.linalg import svd
13 from os import getcwd, mkdir, path
14
15 from PADupdate import PADutil as util
100
16
17 class PADdata(object):
18 """
19 object for functions on 4D self data.
20 """
21 def __init__(self, filename, voltage=300,
22 exposure=1, savepath=getcwd()):
23 """
24 Initializes EMPAD data object by reading in the .raw datafile
25 located at path given by filename string.
26
27 Inputs:
28 filename string indicating path to .raw file from self.
29 Suggest convention with number as beginning of
30 filename, as anything before the first
31 underscore is used as an identifier
32 voltage The microscope voltage during data acquisition:
33 used to set counts per electron in the code.
34 exposure The exposure time in ms, defaults to 1 because
35 when this code was written that was the only
36 option available
37 savepath Where to put output files if not the current
38 directory. Useful if you want to save in a
39 different directory.
40 """
41 #read in the data and crop the "nonsense pixels" from CBEDs
42 self.data = util.readPAD(filename)[1:-2, 3:-2, :, :]
43 #assign dimenstions as attributes for easy reference
44 (self.cbedx, self.cbedy,
45 self.scanx, self.scany) = np.shape(self.data)
46 #define the counts per electron for determining dose rates
47 if voltage == 300:
48 self.countspere = 579
49 else:
50 self.countspere = input("Please input the counts per "
51 "electron at the accelerating voltage used. ")
52 #define the exposure time in seconds
53 self.exposure = exposure/1000 #convert ms to s
54 #keep filename without extension for saving later, if necessary
55 self.f = filename.split('/')[-1]
56 #assign the dataset a "number" from the filename, for
57 #identification in methods where data is autosaved
58 self.number = self.f.split('_')[0]
59 self.savepath = savepath + '/'
60 return
61
62 def scan_crop(self, xmin, xmax, ymin, ymax):
63 """ Crop the data in the scan dimension """
64 self.data = self.data[:,:,xmin:xmax, ymin:ymax]
65 #redefine the data dimensions
66 (self.cbedx, self.cbedy,
67 self.scanx, self.scany) = np.shape(self.data)
101
68 return
69
70 def cbed_crop(self, xmin, xmax, ymin, ymax):
71 """ Crop the data in the CBED dimension """
72 self.data = self.data[xmin:xmax, ymin:ymax, :, :]
73 #redefine the data dimensions
74 (self.cbedx, self.cbedy,
75 self.scanx, self.scany) = np.shape(self.data)
76 return
77
78 def get_cbedsum(self):
79 """Finds sum of all the CBED patterns in the dataset."""
80 self.cbedsum = np.sum(np.sum(self.data, axis=2), axis=2)
81 return
82
83 def get_cbedmean(self):
84 """Finds mean of all the CBED patterns in the dataset."""
85 self.cbedmean = np.mean(np.mean(self.data, axis=2), axis=2)
86 return
87
88 def get_center(self, show=False):
89 """
90 Finds center of CBED pattern by thresholding intensity,
91 then assuming circular shape. Works well for amorphous bio
92 data, not tested on anything crystalline.
93 Option "show" allows the user to verify that the circular
94 region defined by thresholding is reasonable.
95 """
96 def center(input, threshold):
97 """
98 Function to minimize to find center (x,y) of
99 thresholded region.
100 """
101 sigma = sum([(x-input[0])**2+(y-input[1])**2 for x,y in
102 np.swapaxes(np.nonzero(threshold), 0, 1)])
103 return sigma
104 #Find sum of CBEDs and set threshold for bright region:
105 self.get_cbedsum()
106 cbedsum = self.cbedsum
107 threshold = threshold_isodata(cbedsum)
108 mask = cbedsum > threshold
109 if show:
110 util.show(mask)
111 util.show(cbedsum)
112 #Find the center (the point within the region at minimum
113 #distance to all other points)
114 self.cbedcenterx, self.cbedcentery = minimize(center,
115 [self.cbedx/2, self.cbedy/2], args = mask).x
116 #Find the radius (by assuming the area of the region is a circle)
117 self.cbedrad = np.sqrt(np.sum(mask.astype('int'),
118 axis=None)/np.pi)
119 return
102
120
121 def auto_scan_crop(self, image='incobf'):
122 """
123 Looks for dark columns caused by blanking the beam in cryo-imaging.
124 inputs:
125 image Specify whether to use the ADF or the BF image for
126 cropping. Usually bf works, but sometimes if it fails
127 ADF might be better.
128 """
129 #get a scanned image to use to find dark columns
130 if image=='incobf':
131 self.get_incobf()
132 imarr = self.incobf
133 elif image=='adf':
134 self.get_adf()
135 imarr = self.adf
136 elif image=='bf':
137 self.get_bf()
138 imarr = self.bf
139 else:
140 print("Specify image type 'incobf', 'bf', or 'adf'.")
141 return
142 threshold = np.min(imarr)+np.std(imarr)
143 #find columns where many pixels are dark
144 #colsnum is a 1D array giving number of dark pixels per column
145 colsnum = np.sum(imarr<threshold, axis=0)
146 colsmask = colsnum > 0
147 #defining columns to discard in cols
148 cols = np.zeros_like(colsmask)
149 #start from column zero - we'll only discard if columns
150 #adjacent to edges
151 i = 0
152 while colsmask[i]:
153 cols[i] = True
154 i = i+1
155 #and the same from the rightmost column:
156 i = self.scany-1
157 while colsmask[i]:
158 cols[i] = True
159 i = i-1
160 self.data = self.data[:,:,:,˜cols]
161 (self.cbedx, self.cbedy,
162 self.scanx, self.scany) = np.shape(self.data)
163 print("Cropped {} columns".format(np.count_nonzero(cols)))
164 "Regenerate original image with cropped data"
165 if image=='incobf':
166 self.get_incobf()
167 elif image=='adf':
168 self.get_adf()
169 elif image=='bf':
170 self.get_bf()
171 return
103
172
173 def bin_cbeds(self, factor):
174 """
175 bin dataset in CBED space. Useful for registering extremely low dose
176 data when using tilt-corrected bright field, or simple reduction in data
177 size that preserves real-space pixel size.
178
179 Inputs:
180 factor factor by which to bin the CBEDs
181
182 Returns:
183 binned array
184
185 """
186 #shape array to be 3D stack of CBEDs (can then pass to rebin2D)
187 cbedstack = np.reshape(self.data, (self.cbedx, self.cbedy,
188 self.scanx*self.scany))
189 binnedstack = util.rebin2D(cbedstack, factor)
190 #reshape to 4D
191 self.data = np.reshape(binnedstack, (self.cbedx//factor,
192 self.cbedy//factor,
193 self.scanx, self.scany))
194 #update the data dimensions, and the center position as
195 #both have changed
196 (self.cbedx, self.cbedy,
197 self.scanx, self.scany) = np.shape(self.data)
198 self.get_center()
199 return
200
201 def make_mask(self, radius=1):
202 """
203 Make a circular mask in CBED space, for defining bright
204 field image regions.
205
206 Inputs:
207 radius float/integer/number multiplying the CBED radius
208 from get_center()
209 """
210 xgrid, ygrid = np.meshgrid(range(self.cbedx), range(self.cbedy))
211 xgrid = xgrid-self.cbedcenterx
212 ygrid = ygrid-self.cbedcentery
213 rad = np.transpose(np.sqrt(ygrid**2+xgrid**2))
214 outer = self.cbedrad*radius
215 mask = rad < outer
216 return mask
217
218 def find_binning(self, min_electrons=250000):
219 """
220 Determines the amount by which to bin in order to retain a minimum
221 number of electrons in each bright field pixel. Useful for automating
222 tcBF-STEM on low-dose samples.
223
104
224 Inputs:
225 The data array
226 min_electrons integer, describing the minimum amount of electrons
227 (not counts!) desired in each frame
228 Returns:
229 binf integer, describing by how much the data should be bin
230
231 """
232 self.get_center()
233 squaremask = self.make_mask(radius = 0.75)
234 mean = np.mean(np.sum(np.sum(self.data[squaremask.astype(bool)],
235 axis=-1), axis=-1))/579
236 if min_electrons < mean:
237 return 1
238 else:
239 binf = int(np.sqrt(min_electrons/mean))+1
240 return binf
241
242 def center_cbeds(self):
243 """
244 This function corrects for large shifts in the CBEDs as a
245 function of probe position (linear in scanx, scany coordinates)
246 This occurs with misalignments of the scan pivot points, or
247 is simply more noticeable when scanning over large fields
248 of view.
249 """
250 self.get_center()
251 #Find center-of-mass image
252 self.get_com()
253 #get the mean of COM components: x and y relative to x, y coords
254 #Fit the means as a function of coord to a line
255 xymeans = np.mean(self.comx, axis=0)
256 xypopt, xypcov = curve_fit(util.line, range(self.scany), xymeans)
257 xxmeans = np.mean(self.comx, axis=1)
258 xxpopt, xxpcov = curve_fit(util.line, range(self.scanx), xxmeans)
259 yxmeans = np.mean(self.comy, axis=1)
260 yxpopt, yxpcov = curve_fit(util.line, range(self.scanx), yxmeans)
261 yymeans = np.mean(self.comy, axis=0)
262 yypopt, yypcov = curve_fit(util.line, range(self.scany), yymeans)
263 #Define a list of x, y, shifts that will correct any
264 #linear dependence found
265 self.cbedshiftsx = np.zeros_like(self.com)
266 self.cbedshiftsy = np.zeros_like(self.com)
267 for i in range(self.scanx):
268 for j in range(self.scany):
269 self.cbedshiftsx[i,j] = (xypopt[0]*j+xypopt[1]
270 + xxpopt[0]*i+xxpopt[1])
271 self.cbedshiftsy[i,j] = (yxpopt[0]*i+yxpopt[1]
272 + yypopt[0]*j+yypopt[1])
273 #Apply the shifts to sub-pixel accuracy
274 self.data[:,:,i,j] = util.fftshift(self.data[:,:,i,j],
275 -self.cbedshiftsx[i,j],
105
276 -self.cbedshiftsy[i,j])
277 #Crop the CBEDs to prevent any wraparound from shifting
278 xcrop = int(np.max(abs(self.cbedshiftsx)))+1
279 ycrop = int(np.max(abs(self.cbedshiftsy)))+1
280 self.cbed_crop(xcrop, -xcrop, ycrop, -ycrop)
281 #Get the following attributes with the shifted CBEDs:
282 self.get_cbedsum()
283 self.get_cbedmean()
284 self.get_center()
285 self.get_com()
286 return
287
288 def subtract_background(self, threshold=20):
289 """
290 Removes intensity offset of each CBED pattern. Iterating over scan
291 pixels, histogram is fit to Gaussian near zero. The center of the
292 distribution is subtracted.
293
294 Inputs:
295 threshold defines maximum intensity up to which data histogram is fit
296
297 Returns:
298 Array with data at each scan pixel shifted to make them have the same
299 zero value
300 """
301 def gauss1d(x, amplitude, x0, sigma_x, offset):
302 # Returns result as a 1D array that can be passed to
303 #scipy.optimize.curve_fit
304 x0 = float(x0)
305 g = offset+amplitude*np.exp(-1/(2*sigma_x**2)*(x-x0)**2)
306 return g
307 self.background = np.zeros((self.scanx, self.scany))
308 #iterate over image pixels
309 for i in range(self.scanx):
310 for j in range(self.scany):
311 try:
312 #Fit the data below the threshold to a gaussian
313 hist, bins = np.histogram((self.data[:,:,i,j]
314 [self.data[:,:,i,j]<threshold]), bins=128)
315 p0=[np.max(hist),
316 np.mean(self.data[:,:,i,j][self.data[:,:,i,j]
317 <threshold]),
318 np.std(self.data[:,:,i,j][self.data[:,:,i,j]
319 <threshold]), 0]
320 popt, pcov = curve_fit(gauss1d, bins[:-1], hist, p0)
321 #subtract the offset of the data
322 self.data[:,:,i,j] = self.data[:,:,i,j] - popt[1]
323 self.background[i,j] = popt[1]
324 except:
325 #If this doesn't work, forget the fitting
326 #and use the mean
327 print("fitting failed at pixel {}, {};".format(i,j)
106
328 " using mean of data < 10")
329 self.data[:,:,i,j] = (self.data[:,:,i,j]
330 - (np.mean(self.data[:,:,i,j]
331 [self.data[:,:,i,j])<10]))
332 self.background[i,j] = np.mean(self.data[:,:,i,j]
333 [self.data[:,:,i,j]<10])
334 return
335
336 def subtract_background_image(self, background):
337 """
338 A function which you hopefully do not have to use.
339 If for some reason, the detector's automatic background
340 subtraction has failed, this function will subtract a
341 background image from every CBED image.
342
343 Good ways to get the background are to save a small scan with
344 the beam blanked, or to look for regions of blanked data (often
345 happens in cryo imaging) where an average image can be extracted.
346
347 Inputs:
348 background A 2D array of size cbedx, cbedy to subtract from
349 each CBED image
350 """
351 for i in range(self.scanx):
352 for j in range(self.scany):
353 self.data[:,:,i,j] = self.data[:,:,i,j] - background
354 return
355
356 def threshold_dark(self, threshold=15):
357 #removes small/negative counts from dataset
358 mask = self.data < threshold
359 self.data[mask] = 0.0
360
361 #get the sum again with these fixed
362 self.get_cbedsum()
363
364 #Total dose in number of electrons
365 self.dose = np.sum(self.data, axis=None)/self.countspere
366 #Probe current, in Amperes
367 self.current = (self.dose * 1.60217646E-19 /
368 (self.scanx*self.scany*self.exposure))
369 return
370
371 def get_adf(self):
372 """
373 Generate an incoherent ADF image with radius from 3x CBED disk
374 radius to the maximum allowed by the detector.
375 """
376 self.get_cbedsum()
377 if not hasattr(self, 'cbedrad'):
378 self.get_center()
379 self.adf = np.zeros((self.scanx, self.scany))
107
380 xgrid, ygrid = np.meshgrid(range(self.cbedx), range(self.cbedy))
381 xgrid = xgrid-self.cbedcenterx
382 ygrid = ygrid-self.cbedcentery
383 rad = np.transpose(np.sqrt(ygrid**2+xgrid**2))
384 inner = self.cbedrad * 3
385 outer = min(self.cbedx-self.cbedcenterx, self.cbedcenterx,
386 self.cbedy-self.cbedcentery, self.cbedcentery)
387 mask = (rad >= inner) * (rad < outer)
388 for i in range(self.scanx):
389 for j in range(self.scany):
390 self.adf[i,j] = np.sum(self.data[:,:,i,j][mask])
391 self.adfinner = inner
392 self.adfouter = outer
393 return
394
395 def get_bf(self):
396 """
397 Generate a bright-field image based on a detector subtending 1/3
398 the convergence angle in CBED space, centered on the BF disk
399 """
400 if not hasattr(self, 'cbedrad'):
401 self.get_center()
402 mask = self.make_mask(radius=1/3)
403 self.bfrad = 1/3*self.cbedrad
404 self.bf = np.sum(self.data[mask, :, :], axis=0)
405 return
406
407 def get_incobf(self):
408 """
409 Generate an incoherent bright-field image based on a detector
410 subtending the full convergence angle in CBED space, centered
411 on the BF disk
412 """
413 if not hasattr(self, 'cbedrad'):
414 self.get_center()
415 mask = self.make_mask(radius=1)
416 self.incobf = np.sum(self.data[mask, :, :], axis=0)
417 return
418
419 def get_com(self):
420 """
421 Generate a center of mass image.
422
423 For each scan pixel, we find the position of the center of
424 mass relative to the center of the CBED patterns found
425 in get_center.
426
427 Returns:
428 self.comx x offset from center, 2D array with scan dimensions
429 self.comy y offset from center, 2D array with scan dimensions
430 self.com magnitude of shifts from x, y images
431 """
108
432 self.get_cbedsum()
433 self.get_center()
434 self.comx = np.zeros((self.scanx, self.scany))
435 self.comy = np.zeros((self.scanx, self.scany))
436 self.com = np.zeros((self.scanx, self.scany))
437 xgrid, ygrid = np.meshgrid(range(self.cbedx), range(self.cbedy))
438 xgrid = xgrid-self.cbedcenterx
439 ygrid = ygrid-self.cbedcentery
440 for i in range(self.scanx):
441 #print "scan row {}".format(i)
442 for j in range(self.scany):
443 tot = np.sum(self.data[:,:,i,j])
444
445 if tot==0:
446 self.comx[i,j] = 0
447 self.comy[i,j] = 0
448
449 else:
450 self.comx[i,j] = np.sum(self.data[:,:,i,j]
451 * xgrid.T)/tot
452 self.comy[i,j] = np.sum(self.data[:,:,i,j]
453 * ygrid.T)/tot
454 self.com[i,j] = np.sqrt(self.comx[i,j]**2
455 + self.comy[i,j]**2)
456 return
457
458
459 def get_dpc(self, angle=0):
460 """
461 Create a DPC image from the BF disk, by subtracting one
462 half from the other.
463 Halves are specified by "angle" input.
464 Input:
465 angle integer, in degrees. Specifies at which angle to slice
466 the central disk (so unlike fixed detector where we have
467 left/right or up/down, we can slice at any angle
468 we want)
469 """
470 self.dpcangle = np.pi / 180 * angle
471 def dpc_maker(self, angle):
472 '''
473 #helper function for finding DPC images
474 '''
475 angle = np.pi / 180 * angle
476 if not hasattr(self, 'cbedrad'):
477 self.get_center()
478
479 outer = self.cbedrad
480 #define one array based giving radius from center, and one
481 #giving angle from positive x-axis in radians
482 xgrid, ygrid = np.meshgrid(range(self.cbedx),
483 range(self.cbedy))
109
484 xgrid = xgrid-self.cbedcenterx
485 ygrid = ygrid-self.cbedcentery
486 radius = np.transpose(np.sqrt(ygrid**2+xgrid**2))
487 anglegrid = np.transpose(np.arctan2(ygrid,xgrid))
488 #Create masks defining the two halves of the BF disk
489 mask1 = (radius < outer) * (anglegrid < angle)
490 * (angle-np.pi < anglegrid)
491 mask2 = (radius < outer) * (anglegrid > angle)
492 * (angle-np.pi > anglegrid)
493 #subtract the two masks from each other
494 imarray = (self.data[mask1, :, :].sum(axis=0)
495 -self.data[mask2, :, :].sum(axis=0))
496 /np.sum(self.data)
497 return imarray, outer
498 self.dpc, self.dpcouter = dpc_maker(self, angle)
499 self.dpcperp, self.dpcouter = dpc_maker(self, angle+90)
500 self.dpcmag = np.sqrt(self.dpc**2 + self.dpcperp**2)
501 return
502
503 def get_firstmoment(self):
504 """
505 Generates a first-moment image: each pixel in returned
506 image is first moment (in x, y) of intensity distribution
507 of CBED at that scan pixel.
508 """
509 self.firstmomx = np.zeros((self.scanx, self.scany))
510 self.firstmomy = np.zeros((self.scanx, self.scany))
511 #coordinates by which to weight the intensity
512 vx = np.arange(0,self.cbedx)
513 vy = np.arange(0,self.cbedy)
514 #calculate first moment at each scan pixel.
515 for i in range(self.scanx):
516 for j in range(self.scany):
517 cbed = self.data[:,:,i,j]
518 pnorm = np.sum(cbed)
519 if pnorm == 0:
520 self.firstmomx[i,j] = 0
521 self.firstmomy[i,j] = 0
522 else:
523 self.firstmomx[i,j] = np.sum(vx
524 * np.sum(cbed, axis=0), axis=None)/pnorm
525 self.firstmomy[i,j] = np.sum(vy
526 * np.sum(cbed, axis=1), axis=None)/pnorm
527 #combine components. Can easily also output a phase map
528 self.firstmommag = np.sqrt(self.firstmomx**2 + self.firstmomy**2)
529 return
530
531 def get_secondmoment(self):
532 """
533 Generates a second-moment image: each pixel in returned image is first
534 moment (in x, y) of intensity distribution of CBED at that scan pixel.
535 """
110
536 self.secondmomx = np.zeros((self.scanx, self.scany))
537 self.secondmomy = np.zeros((self.scanx, self.scany))
538 if not hasattr(self, 'firstmomx'):
539 self.get_firstmoment()
540 #coordinates by which to weight intensity
541 vx = np.arange(0,self.cbedx)
542 vy = np.arange(0,self.cbedy)
543 #calculate second moment at each pixel using result of first moment
544 for i in range(self.scanx):
545 for j in range(self.scany):
546 cbed = self.data[:,:,i,j]
547 pnorm = np.sum(cbed)
548 if pnorm == 0:
549 self.secondmomx[i,j] = 0
550 self.secondmomy[i,j] = 0
551 else:
552 self.secondmomx[i,j] = np.sum((vx - self.firstmomx[i,j])**2
553 * np.sum(cbed, axis=0), axis=None)/pnorm
554 self.secondmomy[i,j] = np.sum((vy - self.firstmomy[i,j])**2
555 * np.sum(cbed, axis=1), axis=None)/pnorm
556 self.secondmommag = np.sqrt(self.secondmomx**2 + self.secondmomy**2)
557 return
558
559 def get_tcBF(self, radius = 0.8, expand=4, n = 0, correlationType='cc',
560 findMaxima = 'pixel', show=False, crop=True):
561 """
562 Function to get tilt-corrected bright field image by cross correlating
563 images from each BF CBED pixel
564
565 Uses RigidRegistration code by Ben Savitzky:
566 https://github.com/bsavitzky/rigidRegistration/
567
568 Inputs:
569 radius fraction of cbedrad to use for the reconstructed
570 image
571 expand amount by which to upsample (1 scan pixel becomes
572 4x4 default)
573 n Integer, describing falloff of low-pass filter. if
574 zero, no filtering used
575 correlationType passed to findImageShifts; cc, pc, or mc. cc is
576 regular cross-correlation, pc is phase
577 correlation, mc is mutual correlation. See Ben's
578 publication for details.
579 findMaxima Method/precision with which to find maximum in
580 cross-correlations. Pixel precision is usually
581 sufficient, with sub-pixel precision coming from
582 averages of the many images in our BF disk/stack.
583 Other options are "gf" for Gaussian fitting of the
584 cross correlation peak, or "com" for center of mass.
585 show Boolean: with True some intermediate images are
586 output as well as verbose correlations
587 crop Boolean: Crops regions of wraparound from image
111
588 shifting. There shouldn't usually be a reason to
589 turn this off.
590
591 """
592 #Find the center, and define the region of CBED pixels used
593 #for the image
594 if not hasattr(self, 'cbedrad'):
595 self.get_center()
596 self.tcmask = self.make_mask(radius = radius)
597 self.bfreconrad = radius*self.cbedrad
598 #Initialize imstack object from RigidRegistraion using BF CBED
599 #region
600 stack = util.make_stack(np.swapaxes(np.reshape(self.data,
601 (self.cbedx*self.cbedy, self.scanx, self.scany)),
602 0, 2)[:,:,(self.tcmask).flatten()])
603 #calculate all FFTs. The code applies sinˆ2 window in real space
604 stack.getFFTs()
605 #Apply low-pass filter, if desired
606 if n==0:
607 stack.makeFourierMask(mask='none')
608 else:
609 stack.makeFourierMask(mask='lowpass', n=n)
610
611 #Save the dose, for easy comparison to other imaging methods
612 self.bfrecondose = np.sum(stack.imstack, axis=None)
613
614 #Get all the cross correlations and find the maxima by method
615 #specified in function call
616 stack.findImageShifts(correlationType=correlationType,
617 findMaxima=findMaxima, verbose=False)
618 if show:
619 util.show_sample_ccs(stack, correlationType)
620 #We are not using any corrections to the shift matrix here,
621 #but check the rigidregistration code because this can be useful
622 #for low-dose data I've in the past discarded shifts that are
623 #outliers of the full ensemble of entries in the Xij, Yij matrix
624 self.Rij_mask = np.ones((stack.nz,stack.nz))
625 #Find the average shifts
626 stack.get_imshifts()
627 #assign the shift matrices to attributes of PADdata for
628 #easy reference
629 self.xshiftmatrix = stack.X_ij
630 self.yshiftmatrix = stack.Y_ij
631 self.stack = stack.imstack
632 self.tcbf = np.flipud(np.rot90(util.apply_shifts_expand(stack,
633 expand)))
634 xshiftmap = np.copy(self.tcmask).astype('float')
635 yshiftmap = np.copy(self.tcmask).astype('float')
636 xshiftmap[np.nonzero(xshiftmap)] = stack.shifts_x
637 yshiftmap[np.nonzero(yshiftmap)] = stack.shifts_y
638 self.xshiftmap = xshiftmap
639 self.yshiftmap = yshiftmap
112
640 if show:
641 print("X shift matrix")
642 util.show_map(self.xshiftmatrix)
643 print("Y shift matrix")
644 util.show_map(self.yshiftmatrix)
645 print("X shift map")
646 util.show_map(self.xshiftmap)
647 print("Y shift map")
648 util.show_map(self.yshiftmap)
649 util.show(self.tcbf, figsize=(10,10))
650 return
651
652 def PCA(self):
653 """
654 Code to perform PCA on the 4D dataset - looks at scan images
655 relative to CBED components.
656
657 Can give some cool results but I haven't used it much on bio
658 images - KAS
659 """
660
661 #reshape data into 2D array, with each row being one CBED and
662 #each column one scan image
663 SI = self.data.reshape(self.cbedx*self.cbedy,
664 self.scanx*self.scany)
665 meanim = np.mean(SI, axis=1)
666 meancbed = np.mean(SI, axis=0)
667 gain = 1
668 #normalization factors calculation
669 g = 1/np.sqrt(meanim)
670 h = 1/np.sqrt(meancbed+meancbed**2*2*gain**2)
671 gg = g/np.sum(g, axis=None)
672 hh = h/np.sum(h, axis=None)
673 SI = (hh*(gg*SI.T).T)
674 #perform SVD using scipy.linalg.svd
675 print("doing singular value decomposition")
676 u,s,v = svd(SI, full_matrices=False)
677 #recombine to get images per component
678 scoresweight = np.dot(np.diag(np.sqrt(1/gg)),np.dot(u,
679 np.diag(s)))
680 loadsweight = np.dot(np.dot(np.diag(s), v),
681 np.diag(np.sqrt(1/hh)))
682 #Sscores are CBEDs, loads are scan images.
683 self.scores = scoresweight.reshape((self.cbedx, self.cbedy, -1),
684 order ='F')
685 self.loads = np.swapaxes(loadsweight.reshape((-1, self.scany,
686 self.scanx), order='F'), 0, 2)
687 return
A second helper file holds some of the functions required in PAD.py.
113
1 """
2 PADutil.py: helper functions and RigidRegistration usage for PAD.py
3
4 This version: September 2018, Katherine Spoth
5 """
6 from __future__ import division, print_function
7 import numpy as np
8 from scipy.optimize import minimize, curve_fit
9 from scipy.linalg import svd
10 from os import getcwd, mkdir, path
11 import matplotlib.pyplot as plt
12 from PIL import Image, ImageDraw
13 import tifffile
14 from rigidRegistration.rigidregistration import stackregistration as stackreg
15 #Ben Savitzky; https://github.com/bsavitzky/rigidRegistration/
16
17 def readPAD(filename):
18 """
19 Function to read in data from the EMPAD.
20
21 Returns the data in 4D array (cbed x, cbed y, scan x, scan y)
22 of 32-bit floats.
23 """
24 #A clumsy way to get the scan dimensions from the default filenames:
25 scanx = int(filename.rstrip('.raw').split('_')[-1].lstrip('xy'))
26 scany = int(filename.rstrip('.raw').split('_')[-2].lstrip('xy'))
27 #open the file
28 contents = np.fromfile(filename, dtype = 'float32')
29 #reshape and return
30 return np.reshape(contents, (128, 130, scany, scanx), order = 'F')
31
32 def line(x, slope, intercept):
33 """
34 A line, used to find misalignments in CBED as function of scan
35 position
36 (This happens often with very large fields of view - scan pivot
37 point alignment)
38 """
39 return slope*x + intercept
40
41 def fftshift(array, xshift, yshift):
42 """
43 A function to apply sub-pixel shifts to an image, using Fourier
44 space.
45
46 inputs:
47 array: the 2D image
48 xshift, yshift: desired shift amount in fractional pixel value
49 Returns: the shifted array
50 """
51 #create image coordinate grid
52 rx, ry = np.meshgrid(np.arange(np.shape(array)[0]),
114
53 np.arange(np.shape(array)[1]))
54 x, y = float(np.shape(array)[0]), float(np.shape(array)[1])
55 #shift in Fourier space:
56 w = -np.exp(-(2j*np.pi)*(xshift*rx/x+yshift*ry/y))
57 shifted_fft = np.fft.fftshift(np.fft.fft2(array))*w.T
58 # inverse transform back to real space and return
59 return np.abs(np.fft.ifft2(np.fft.ifftshift(shifted_fft)))
60
61 def rebin2D(array, factor):
62 """
63 Re-bin array by specified factor.
64 Inputs
65 array Data to bin
66 factor Amount to bin by, such that the new array shape is the
67 original's shape modulo the factor.
68 Returns:
69 binned array
70 """
71 shape = np.shape(array)
72 #for 3 dimensional array, bin each z-slice
73 if len(shape) == 3:
74 # #ensure even dimension
75 newshape = (shape[0]//factor), (shape[1]//factor), shape[2]
76 #remove pixels that we won't use
77 arraycrop = array[0:(newshape[0]*factor),0:(newshape[1]*factor), :]
78 #define array to write output into
79 binned = np.zeros(newshape)
80 #bin each slice and save
81 for i in range(shape[2]):
82 binned[:,:,i] = arraycrop[:,:,i].reshape([newshape[0], factor,
83 newshape[1], factor]).sum(-1).sum(1)
84 #2 dimensional array: same handling, but one slice
85 else:
86 newshape = shape[0]//factor, shape[1]//factor
87 arraycrop = array[0:(newshape[0]*factor),0:(newshape[1]*factor)]
88 binned = arraycrop.reshape([newshape[0], factor, newshape[1],
89 factor]).sum(-1).sum(1)
90
91 return binned
92
93 def radius_grid(array):
94 """
95 Function to make an array the shape of input array; each pixel is
96 valued its radius from the center.
97 """
98 a, b = np.shape(array)
99 xgrid, ygrid = np.meshgrid(range(a), range(b))
100 xgrid = xgrid-a/2
101 ygrid = ygrid-b/2
102 rad = np.transpose(np.sqrt(xgrid**2+ygrid**2))
103 return rad
104
115
105 def low_pass_window(array, b):
106 """
107 Defines window that low-pass filters an array when applied in
108 Fourier space
109 b is a real space size in pixels, b/FOV describes the 1/e
110 fall-off in Fourier space.
111 """
112 r = max(np.shape(array))
113 rad = radius_grid(array)
114 weight = 10**(-rad**2/(r/b)**2)
115 return weight
116
117 def low_pass_filter(array, b):
118 """
119 Actually filters the array, see low_pass_window for math
120 """
121 def lp(array, b):
122 window = low_pass_window(array, b)
123 return abs(np.fft.ifft2(window*np.fft.fftshift(np.fft.fft2(array))))
124 if array.ndim == 2:
125 return lp(array, b)
126 elif array.ndim == 3:
127 filtered = np.zeros_like(array)
128 for i in range(np.shape(array)[-1]):
129 filtered[:,:,i] = lp(array[:,:,i], b)
130 return filtered
131
132 ######
133 # Registration functions for tilt corrected BF. Utilize Ben's
134 # RigidRegistration code.
135
136 def make_stack(array):
137 #make imstack object
138 return stackreg.imstack(array)
139
140 def apply_shifts_expand(imstack, expand):
141 """
142 Upsample images by expand using nearest-neighbor; apply the shifts
143 after expanding.
144 Inputs:
145 imstack Registration.imstack object. Needs to have average
146 shifts already calculated
147 expand Integer, how much by which to expand the image.
148 """
149 imstack.stack_registered=np.zeros((expand*imstack.nx,
150 expand*imstack.ny,imstack.nz))
151 for z in range(imstack.nz):
152 im = imstack.imstack[:,:,z]
153 expim = np.zeros((expand*imstack.nx, expand*imstack.ny))
154 for i in range(imstack.nx*expand):
155 for j in range(imstack.ny*expand):
156 expim[i,j] = im[i//expand, j//expand]
116
157 imstack.stack_registered[:,:,z] = stackreg.generateShiftedImage(expim,
158 imstack.shifts_x[z]*expand,
159 imstack.shifts_y[z]*expand)
160 imstack.average_image = np.sum(imstack.stack_registered,axis=2)
161 return imstack.average_image
162
163 def show_sample_ccs(imstack, correlationType):
164 if correlationType=="cc":
165 getSingleCorrelation = imstack.getSingleCrossCorrelation
166 elif correlationType=="mc":
167 getSingleCorrelation = imstack.getSingleMutualCorrelation
168 elif correlationType=="pc":
169 getSingleCorrelation = imstack.getSinglePhaseCorrelation
170 fft1 = imstack.fftstack[:,:,0]
171 fft2 = imstack.fftstack[:,:,imstack.nz//2]
172 fft3 = imstack.fftstack[:,:,-1]
173 print("A few cross-correlations:")
174 show(np.abs(np.fft.fftshift((getSingleCorrelation(fft1, fft2)))),
175 cmap='jet')
176 show(np.abs(np.fft.fftshift((getSingleCorrelation(fft1, fft3)))),
177 cmap='jet')
178 show(np.abs(np.fft.fftshift((getSingleCorrelation(fft2, fft3)))),
179 cmap='jet')
180 return
181
182 ######
183 #Handy functions for displaying, so you don't have to remember
184 #lines and lines of pyplot code
185
186 def show(data, cmap='gray', **kwargs):
187 """
188 Show an image using pyplot. Removes frames, ticks, etc so plot looks
189 like a normal image. Very useful for jupyter notebooks.
190 Inputs:
191 Data 2D image array to show
192 cmap The colormap for displaying the data. greyscale by default.
193 **kwargs Passed to the figure() function - useful for changing size,
194 dpi, etc
195 """
196 fig = plt.figure(**kwargs)
197 plt.matshow(data, fignum = False, cmap = cmap)
198 a=fig.gca()
199 a.set_frame_on(False)
200 a.set_xticks([]); a.set_yticks([])
201 plt.axis('off')
202 plt.show()
203 return
204
205 def show_map(array, cmap='bwr', v=False, vmin=False, vmax=False, **kwargs):
206 """
207 Similar to show above, but more useful for data other than images.
208 I use it for the shift matrices and maps; colormap defaults to be
117
209 symmetric about zero (eg Most red (positive) shifts are equivalent
210 to most blue (negative) shifts).
211 Inputs:
212 array 2D array to display
213 cmap colormap to display the data - default is bwr, any
214 white-centered map is sensible for shift data
215 v cmap limits: set max as +v and min as -v automatically
216 vmax, vmin useful if user wants asymmetric data limits
217 **kwargsPassed to plt.figure, useful are figsize=(), etc.
218 """
219 fig = plt.figure(**kwargs)
220 if vmin==False and vmax==False and v==False:
221 v = np.max(abs(array), axis=None)
222 vmin = -v
223 vmax = v
224 if v!= False:
225 vmin = -v
226 vmax = v
227 plt.matshow(array, fignum = False, cmap = cmap,
228 vmin = vmin, vmax = vmax)
229 a=fig.gca()
230 a.set_frame_on(False)
231 a.set_xticks([]); a.set_yticks([])
232 plt.axis('off')
233 plt.colorbar(shrink = 0.65)
234 plt.show()
235 return
236
237 ######
238 #Saving data
239
240 def saveTiff(array, filename, bits=16):
241 t = 'uint{}'.format(bits)
242 scalearr = (imarr - np.min(imarr))/(np.max(imarr)-np.min(imarr))
243 imarr = ((2**bits-1)*scalearr).astype(t)
244 if len(np.shape(imarr))>2:
245 imarr = np.swapaxes(imarr,0,2)
246 tifffile.imsave(filename+'.tif', imarr)
118
REFERENCES
[1] Medalia, O, Weber, I, Frangakis, A. S, Nicastro, D, Gerisch, G, & Baumeis-
ter, W. (2002) Macromolecular architecture in eukaryotic cells visualized
by cryoelectron tomography. Science 298, 1209–1213.
[2] Noble, J. M, Lubieniecki, J, Savitzky, B. H, Plitzko, J, Engelhardt, H,
Baumeister, W, & Kourkoutis, L. F. (2018) Connectivity of centermost
chromatophores in Rhodobacter sphaeroides bacteria. Molecular Micro-
biology 1817, 1616.
[3] Irobalieva, R. N, Martins, B, & Medalia, O. (2016) Cellular structural bi-
ology as revealed by cryo-electron tomography. Journal of Cell Science 129,
469–476.
[4] Cheng, Y, Grigorieff, N, Penczek, P. A, & Walz, T. (2015) A primer to
single-particle cryo-electron microscopy. Cell 161, 438–449.
[5] Patterson, J. P, Xu, Y, Moradi, M.-A, Sommerdijk, N. A. J. M, & Friedrich,
H. (2017) CryoTEM as an advanced analytical tool for materials chemists.
Accounts of Chemical Research 50, 1495–1501.
[6] El Baggari, I, Savitzky, B. H, Admasu, A. S, Kim, J, Cheong, S.-W, Hovden,
R, & Kourkoutis, L. F. (2018) Nature and evolution of incommensurate
charge order in manganites visualized with cryogenic scanning transmis-
sion electron microscopy. Proceedings of the National Academy of Sciences
115, 1445–1450.
[7] Tsen, A. W, Hovden, R, Wang, D, Kim, Y. D, Okamoto, J, Spoth, K. A, Liu,
Y, Lu, W, Sun, Y, Hone, J. C, Kourkoutis, L. F, Kim, P, & Pasupathy, A. N.
(2015) Structure and control of charge density waves in two-dimensional
1T-TaS2. Proceedings of the National Academy of Sciences 112, 15054–15059.
[8] Zachman, M. J, Tu, Z, Choudhury, S, Archer, L. A, & Kourkoutis, L. F.
(2018) Cryo-STEM mapping of solid–liquid interfaces and dendrites in
lithium-metal batteries. Nature 560, 345–349.
[9] Sun, Y, Ma, K, Kao, T, Spoth, K. A, Sai, H, Zhang, D, Kourkoutis, L. F,
Elser, V, & Wiesner, U. (2017) Formation pathways of mesoporous silica
nanoparticles with dodecagonal tiling. Nature Communications 8, 1951.
119
[10] Spoth, K. A, Ma, K, Wiesner, U, & Kourkoutis, L. F. (2016) Understanding
initial formation stages of nanomaterials using cryo-TEM. Microscopy and
Microanalysis 22, 1844–1845.
[11] Ruska, E. (1987) The development of the electron microscope and of elec-
tron microscopy. Reviews of Modern Physics 59, 627–638.
[12] Kirkland, E. J. (2010) Advanced Computing in Electron Microscopy. (Springer
US, Boston, MA).
[13] Grimm, R, Typke, D, B rmann, M, & Baumeister, W. (1996) Determination
of the inelastic mean free path in ice by examination of tilted vesicles and
automated most probable loss imaging. Ultramicroscopy 63, 169–179.
[14] Alamoudi, A, Studer, D, & Dubochet, J. (2005) Cutting artefacts and cut-
ting process in vitreous sections for cryo-electron microscopy. Journal of
Structural Biology 150, 109–121.
[15] Kourkoutis, L. F, Plitzko, J. M, & Baumeister, W. (2012) Electron mi-
croscopy of biological materials at the nanometer scale. Annual review
of Materials Research 42, 33–58.
[16] Yang, P, Gai, S, & Lin, J. (2012) Functionalized mesoporous silica materials
for controlled drug delivery. Chemical Society Reviews 41, 3679–3698.
[17] Dubochet, J, Frank, J, & Henderson, R. (2017) The nobel prize in chemistry
2017.
[18] (2011) Electron microscopy tutorial.
[19] Morán, H. F. (1960) Low-temperature preparation techniques for electron
microscopy of biological specimens based on rapid freezing with liquid
helium II*. Annals of the New York Academy of Sciences 85, 689–713.
[20] Taylor, K. A & Glaeser, R. M. (1976) Electron microscopy of frozen hy-
drated biological specimens. Journal of Ultrastructure Research 55, 448–456.
[21] Dubochet, J, Adrian, M, Chang, J.-J, Homo, J.-C, Lepault, J, McDowall,
A. W, & Schultz, P. (1988) Cryo-electron microscopy of vitrified speci-
mens. Quarterly Reviews of Biophysics 21, 129–228.
120
[22] Moor, H. (1987) in Cryotechniques in Biological Electron Microscopy.
(Springer, Berlin, Heidelberg), pp. 175–191.
[23] Tivol, W. F, Briegel, A, & Jensen, G. J. (2008) An improved cryogen for
plunge freezing. Microscopy and Microanalysis 14, 375–379.
[24] Egerton, R. F, Li, P, & Malac, M. (2004) Radiation damage in the TEM and
SEM. Micron 35, 399–409.
[25] Bammes, B. E, Jakana, J, Schmid, M. F, & Chiu, W. (2010) Radiation dam-
age effects at four specimen temperatures from 4 to 100 K. Journal of Struc-
tural Biology 169, 331–341.
[26] Taylor, K. A & Glaeser, R. M. (2008) Retrospective on the early develop-
ment of cryoelectron microscopy of macromolecules and a prospective on
opportunities for the future. Journal of Structural Biology 163, 214–223.
[27] Henderson, R, Cattermole, D, McMullan, G, Scotcher, S, Fordham, M,
Amos, W. B, & Faruqi, A. R. (2007) Digitisation of electron microscope
films: Six useful tests applied to three film scanners. Ultramicroscopy 107,
73–80.
[28] McMullan, G, Faruqi, A. R, & Henderson, R. (2016) Direct electron detec-
tors. Methods in Enzymology 579, 1–17.
[29] Zheng, S. Q, Palovcak, E, Armache, J.-P, Verba, K. A, Cheng, Y, & Agard,
D. A. (2017) MotionCor2: anisotropic correction of beam-induced motion
for improved cryo-electron microscopy. Nat Meth 14, 331–332.
[30] Savitzky, B. H, El Baggari, I, Clement, C. B, Waite, E, Goodge, B. H, Baek,
D. J, Sheckelton, J. P, Pasco, C, Nair, H, Schreiber, N. J, Hoffman, J, Ad-
masu, A. S, Kim, J, Cheong, S.-W, Bhattacharya, A, Schlom, D. G, Mc-
Queen, T. M, Hovden, R, & Kourkoutis, L. F. (2018) Image registration of
low signal-to-noise cryo-STEM data. Ultramicroscopy 191, 56–65.
[31] Crewe, A. V & Wall, J. (1970) A scanning microscope with 5 å resolution.
Journal of Molecular Biology 48, 375–393.
[32] Crewe, A. V. (1971) High resolution scanning microscopy of biological
specimens. Philosophical Transactions of the Royal Society of London B: Bio-
logical Sciences 261, 61–70.
121
[33] Kellenberger, E, Carlemalm, E, Villiger, W, Wurtz, M, Mory, C, & Colliex,
C. H. (1986) Z-contrast in biology: A comparison with other imaging
modes. Annals of the New York Academy of Sciences 483, 202–228.
[34] Engel, A, Dubochet, J, & Kellenberger, E. (1976) Some progress in the
use of a scanning transmission electron microscope for the observation of
biomacromolecules. Journal of Ultrastructure Research 57, 322–330.
[35] Engel, A. (1978) The STEM: An attractive tool for the biologist. Ultrami-
croscopy 3, 355–357.
[36] Jones, A. V & Leonard, K. R. (1978) Scanning transmission electron mi-
croscopy of unstained biological sections. Nature 271, 659–660.
[37] Carlemalm, E & Kellenberger, E. (1982) The reproducible observation of
unstained embedded cellular material in thin sections: visualisation of an
integral membrane protein by a new mode of imaging for STEM. The
EMBO Journal 1, 63–67.
[38] Hohn, T, Wurtz, M, & Engel, A. (1978) Sevenfold rotational symmetry of
a protein complex. Journal of Ultrastructure Research 65, 90–93.
[39] Ohtsuki, M, Isaacson, M. S, & Crewe, A. V. (1979) Dark field imaging
of biological macromolecules with the scanning transmission electron mi-
croscope. Proceedings of the National Academy of Sciences 76, 1228–1232.
[40] Tichelaar, W, Oostergetel, G. T, Haker, J, Van Heel, M. G, & Van Bruggen,
E. F. J. (1980) Scanning transmission electron microscopy of biological
macromolecules. Ultramicroscopy 5, 27–33.
[41] Engel, A & Reichelt, R. (1984) Imaging of biological structures with the
scanning transmission electron microscope. Journal of Ultrastructure Re-
search 88, 105–120.
[42] Colliex, C. (1986) Electron energy-loss spectroscopy analysis and imaging
of biological specimens. Annals of the New York Academy of Sciences 483,
311–325.
[43] Colliex, C, Mory, C, Olins, A. L, Olins, D. E, & Tencé, M. (1989) Energy
filtered STEM imaging of thick biological sections. Journal of Microscopy
153, 1–21.
122
[44] Leapman, R. D & Andrews, S. B. (1991) Analysis of directly frozen macro-
molecules and tissues in the field-emission STEM. Journal of Microscopy
161, 3–19.
[45] Leapman, R. D & Sun, S. (1995) Cryo-electron energy loss spectroscopy:
observations on vitrified hydrated specimens and radiation damage. Ul-
tramicroscopy 59, 71–79.
[46] Sun, S. Q, Shi, S. L, Hunt, J. A, & Leapman, R. D. (2011) Quantitative wa-
ter mapping of cryosectioned cells by electron energy-loss spectroscopy.
Journal of Microscopy 177, 18–30.
[47] Aoyama, K, Takagi, T, Hirase, A, & Miyazawa, A. (2008) STEM tomogra-
phy for thick biological specimens. Ultramicroscopy 109, 70–80.
[48] Hohmann-Marriott, M. F, Sousa, A. A, Azari, A. A, Glushakova, S, Zhang,
G, Zimmerberg, J, & Leapman, R. D. (2009) Nanoscale 3D cellular imaging
by axial scanning transmission electron tomography. Nat Meth 6, 729–731.
[49] Wolf, S. G, Houben, L, & Elbaum, M. (2014) Cryo-scanning transmission
electron tomography of vitrified cells. Nat Meth 11, 423–428.
[50] Wolf, S. G, Rez, P, & Elbaum, M. (2015) Phosphorus detection in vitrified
bacteria by cryo-STEM annular dark-field analysis. Journal of Microscopy
260.
[51] Elad, N, Bellapadrona, G, Houben, L, Sagi, I, & Elbaum, M. (2017) Detec-
tion of isolated metal ions on ferritin by single-particle cryo-STEM recon-
struction. Microscopy and Microanalysis 23, 802–803.
[52] Kühlbrandt, W. (2014) The resolution revolution. Science 343, 1443–1444.
[53] Kresge, C. T, Leonowicz, M. E, Roth, W. J, Vartuli, J. C, & Beck, J. S.
(1992) Ordered mesoporous molecular sieves synthesized by a liquid-
crystal template mechanism. Nature 359, 710–712.
[54] Tang, F, Li, L, & Chen, D. (2012) Mesoporous silica nanoparticles: Synthe-
sis, biocompatibility and drug delivery. Advanced Materials 24, 1504–1534.
[55] Hoffmann, F, Cornelius, M, Morell, J, & Fröba, M. (2006) Silica-based
mesoporous organic–inorganic hybrid materials. Angewandte Chemie (In-
ternational Ed. in English) 45, 3216–3251.
123
[56] Giraldo, L. F, López, B. L, Pérez, L, Urrego, S, Sierra, L, & Mesa, M. (2007)
Mesoporous silica applications. Macromolecular Symposia 258, 129–141.
[57] Sheng Zhong, D. J. P. (2010) Cryogenic transmission electron microscopy
for direct observation of polymer and small-molecule materials and struc-
tures in solution. Polymer Reviews 50, 287–320.
[58] Newcomb, C. J, Moyer, T. J, Lee, S. S, & Stupp, S. I. (2012) Advances
in cryogenic transmission electron microscopy for the characterization of
dynamic self-assembling nanostructures. Current Opinion in Colloid & In-
terface Science 17, 350–359.
[59] Hickey, R. J, Koski, J, Meng, X, Riggleman, R. A, Zhang, P, & Park, S.-
J. (2014) Size-controlled self-assembly of superparamagnetic polymer-
somes. ACS Nano 8, 495–502.
[60] Elazzouzi-Hafraoui, S, Nishiyama, Y, Putaux, J.-L, Heux, L, Dubreuil, F, &
Rochas, C. (2007) The shape and size distribution of crystalline nanopar-
ticles prepared by acid hydrolysis of native cellulose. Biomacromolecules 9,
57–65.
[61] Balmes, O, Malm, J.-O, Karlsson, G, & Bovin, J.-O. (2005) Cryo-TEM ob-
servation of 3-dimensionally ordered aggregates of 5-nm gold particles in
organic solvents. Journal of Nanoparticle Research 6, 569–576.
[62] Kumar, S, Wang, Z, Penn, R. L, & Tsapatsis, M. (2008) A structural res-
olution cryo-TEM study of the early stages of MFI growth. Journal of the
American Chemical Society 130, 17284–17286.
[63] Yuwono, V. M, Burrows, N. D, Soltis, J. A, & Penn, R. L. (2010) Oriented
aggregation: Formation and transformation of mesocrystal intermediates
revealed. Journal of the American Chemical Society 132, 2163–2165.
[64] Pouget, E. M, Bomans, P. H. H, Goos, J. A. C. M, Frederik, P. M, de With, G,
& Sommerdijk, N. A. J. M. (2009) The initial stages of template-controlled
CaCO3 formation revealed by cryo-TEM. Science 323, 1455–1458.
[65] Baumgartner, J, Dey, A, Bomans, P. H. H, Le Coadou, C, Fratzl, P, Som-
merdijk, N. A. J. M, & Faivre, D. (2013) Nucleation and growth of mag-
netite from solution. Nature Materials 12, 310–314.
124
[66] Carcouët, C. C. M. C, van de Put, M. W. P, Mezari, B, Magusin, P. C. M. M,
Laven, J, Bomans, P. H. H, Friedrich, H, Esteves, A. C. C, Sommerdijk, N.
A. J. M, van Benthem, R. A. T. M, & de With, G. (2014) Nucleation and
growth of monodisperse silica nanoparticles. Nano Letters 14, 1433–1438.
[67] Azzalini, A. (1985) A class of distributions which includes the normal
ones. Scandinavian Journal of Statistics.
[68] Ma, K, Mendoza, C, Hanson, M, Werner-Zwanziger, U, Zwanziger, J, &
Wiesner, U. (2015) Control of ultrasmall sub-10 nm ligand-functionalized
fluorescent core–shell silica nanoparticle growth in water. Chemistry of
Materials 27, 4119–4133.
[69] Ma, K, Spoth, K. A, Cong, Y, Zhang, D, Aubert, T, Turker, M. Z, Kourk-
outis, L. F, Mendes, E, & Wiesner, U. (2018) Early formation pathways
of surfactant micelle directed ultrasmall silica ring and cage structures.
Manuscript in preparation.
[70] Ma, K, Gong, Y, Aubert, T, Turker, M. Z, Kao, T, Doerschuk, P. C, & Wies-
ner, U. (2018) Self-assembly of highly symmetrical, ultrasmall inorganic
cages directed by surfactant micelles. Nature 558, 577–580.
[71] Sun, L, Huang, W. M, Ding, Z, Zhao, Y, Wang, C. C, Purnawali, H, &
Tang, C. (2012) Stimulus-responsive shape memory materials: A review.
Materials & Design 33, 577–640.
[72] Stuart, M. A. C, Huck, W. T. S, Genzer, J, Müller, M, Ober, C, Stamm,
M, Sukhorukov, G. B, Szleifer, I, Tsukruk, V. V, Urban, M, Winnik, F,
Zauscher, S, Luzinov, I, & Minko, S. (2010) Emerging applications of
stimuli-responsive polymer materials. Nature Materials 9, 101–113.
[73] Sun, Y, Sai, H, Spoth, K. A, Tan, K. W, Werner-Zwanziger, U, Zwanziger,
J, Gruner, S. M, Kourkoutis, L. F, & Wiesner, U. (2016) Stimuli-responsive
shapeshifting mesoporous silica nanoparticles. Nano Letters 16, 651–655.
[74] Nadrah, P, Planinšek, O, & Gaberšček, M. (2014) Stimulus-responsive
mesoporous silica particles. Journal of Materials Science 49, 481–495.
[75] Egerton, R. F, Li, P, & Malac, M. (2004) Radiation damage in the TEM and
SEM. Micron 35, 399–409.
125
[76] Kremer, J. R, Mastronarde, D. N, & McIntosh, J. R. (1996) Computer visu-
alization of three-dimensional image data using IMOD. Journal of Struc-
tural Biology 116, 71–76.
[77] (2014) Tomviz for tomographic visualization of nanomaterials [computer
software].
[78] Hovden, R, Hanwell, M. D, Ayachit, U, Jiang, Y, Maynard, R, & Muller,
D. A. (2015) Repeatable and transferable processing for electron tomogra-
phy: An open platform for visualization and reconstruction of 3D materi-
als. Microscopy and Microanalysis 21, 2407–2408.
[79] Hovden, R, Jiang, Y, & Xin, H. L. (2015) Periodic artifact reduction in
Fourier transforms of full field atomic resolution images. Microscopy and
Microanalysis.
[80] Bai, X, McMullan, G, & Scheres, S. H. W. (2015) How cryo-EM is revolu-
tionizing structural biology. Trends in Biochemical Sciences 40, 49–57.
[81] Gan, L & Jensen, G. J. (2012) Electron tomography of cells. Quarterly
Reviews of Biophysics 45, 27–56.
[82] Chen, Z, Weyland, M, Ercius, P, Ciston, J, Zheng, C, Fuhrer, M. S,
D’Alfonso, A. J, Allen, L. J, & Findlay, S. D. (2016) Practical aspects of
diffractive imaging using an atomic-scale coherent electron probe. Ultra-
microscopy 169, 107–121.
[83] Han, Y, Nguyen, K, Cao, M, Cueva, P, Xie, S, Tate, M. W, Purohit, P,
Gruner, S. M, Park, J, & Muller, D. A. (2018) Strain mapping of two-
dimensional heterostructures with subpicometer precision. Nano Letters
18, 3746–3751.
[84] Tate, M. W, Purohit, P, Chamberlain, D, Nguyen, K. X, Hovden, R, Chang,
C. S, Deb, P, Turgut, E, Heron, J. T, Schlom, D. G, Ralph, D. C, Fuchs,
G. D, Shanks, K. S, Philipp, H. T, Muller, D. A, & Gruner, S. M. (2016)
High dynamic range pixel array detector for scanning transmission elec-
tron microscopy. Microscopy and Microanalysis 22, 237–249.
[85] Jiang, Y, Chen, Z, Han, Y, Deb, P, Gao, H, Xie, S, Purohit, P, Tate, M. W,
Park, J, Gruner, S. M, Elser, V, & Muller, D. A. (2018) Electron ptychogra-
phy of 2D materials to deep sub-Angström resolution. Nature 559, 343–
349.
126
[86] Cho, H.-J, Hyun, J.-K, Kim, J.-G, Jeong, H. S, Park, H. N, You, D.-J, &
Jung, H. S. (2013) Measurement of ice thickness on vitreous ice embed-
ded cryo-EM grids: investigation of optimizing condition for visualizing
macromolecules. Journal of Analytical Science and Technology 4, 7.
[87] Hyun, J. K, Ercius, P, & Muller, D. A. (2008) Beam spreading and spatial
resolution in thick organic specimens. Ultramicroscopy 109, 1–7.
[88] Muller, D. A, Nguyen, K. X, Tate, M. W, Purohit, P, Chang, C, Cao, M,
& Gruner, S. M. (2016) An electron microscope pixel array detector as a
universal STEM detector. Microscopy and Microanalysis 22, 478–479.
[89] Ercius, P & Muller, D. A. (2009) Incoherent bright field STEM for imaging
and tomography of ultra-thick TEM cross-sections. Microscopy and Micro-
analysis 15, 238–239.
[90] Zhang, X & Hong Zhou, Z. (2011) Limiting factors in atomic resolution
cryo electron microscopy: No simple tricks. Journal of Structural Biology
175, 253–263.
[91] Koster, A. J & De Ruijter, W. J. (1992) Practical autoalignment of transmis-
sion electron microscopes. Ultramicroscopy 40, 89–107.
[92] Koster, A. J & de Jong, A. F. (1991) Measurement of the spherical aberra-
tion coefficient of transmission electron microscopes by beam-tilt-induced
image displacements. Ultramicroscopy 38, 235–240.
[93] Lupini, A. R, Chi, M, & Jesse, S. (2016) Rapid aberration measurement
with pixelated detectors. Journal of Microscopy 263, 43–50.
[94] Smith, D. J, Saxton, W. O, O’Keefe, M. A, Wood, G. J, & Stobbs, W. M.
(1983) The importance of beam alignment and crystal tilt in high resolu-
tion electron microscopy. Ultramicroscopy 11, 263–281.
[95] Hawkes, P. W. (1980) Electron microscope transfer functions in closed
form with tilted illumination. Optik 55, 207–212.
[96] Erickson, H. P & Klug, A. (1971) Measurement and compensation of de-
focusing and aberrations by fourier processing of electron micrographs.
Philosophical Transactions of the Royal Society of London B: Biological Sciences
261, 105–118.
127
[97] Saxton, W. O & Baumeister, W. (1982) The correlation averaging of a reg-
ularly arranged bacterial cell envelope protein. Journal of Microscopy 127,
127–138.
[98] Downing, K. H. (1991) Spot-scan imaging in transmission electron mi-
croscopy. Science 251, 53–59.
[99] Baker, L. A, Smith, E. A, Bueler, S. A, & Rubinstein, J. L. (2010) The res-
olution dependence of optimal exposures in liquid nitrogen temperature
electron cryomicroscopy of catalase crystals. Journal of Structural Biology
169, 431–437.
[100] Hayward, S. B & Glaeser, R. M. (1979) Radiation damage of purple mem-
brane at low temperature. Ultramicroscopy 4, 201–210.
[101] Pelz, P. M, Qiu, W. X, Bücker, R, Kassier, G, & Miller, R. J. D. (2017) Low-
dose cryo electron ptychography via non-convex Bayesian optimization.
Scientific Reports 7, 9883.
[102] Jiang, Y, Chen, Z, El Baggari, I, Kourkoutis, L. F, Elser, V, & Muller, D. A.
(2018) Breaking the Rayleigh limit in thick samples with multi-slice pty-
chography. Microscopy and Microanalysis 24, 192–193.
128