Non-destructive evaluation of structural components is critical for reducing costs from unnecessary replacements and maintenance. We study the utility of a non-contact modality for the inspection of structural components for the detection and characterization of damage in the form of through cracks and localized corrosion. We focus on the characterization of very small damage with a thermal imaging technique, since sensitivity to early stages of deterioration allows for simpler and less expensive repair than if a flaw propagates and becomes more threatening. The damage we consider interacts with the flow of heat so that a structure's thermal response to a known energy input can provide useful information for inference. Strategies are developed for optimizing a noise-sensitive thermographic experiment to produce optimal data for determining the otherwise hidden properties of the structure. Bayesian inference methods are developed for these tasks, as well as a novel heterogeneous computing method for rapidly simulating the conduction of heat through a three dimensional structure having heterogeneous material properties. Our optimized experiment design for crack characterization is found to produce the same quality of inference as previous settings with much more expensive equipment (e.g. powerful lasers and sensitive IR cameras). It is also found that detection and inference can be done on corrosion pits only millimeters deep in the rear side of a steel panel using thermal observations from the front side.