Interactive Knowledge Extraction and Learning for Architecting Complex Systems
Design space exploration is a popular approach to tackle early-phase engineering design problems, as it presents an opportunity for decision makers to discover important hidden aspects of the problem. In order to help designers gain useful insights, various knowledge discovery tools have been introduced. One popular approach is to use data mining algorithms to extract design knowledge in the form of logical IF-THEN rules. These methods utilize automated search algorithms to extract patterns from data. As the main goal of utilizing data mining algorithms during design space exploration is to help designers gain new insights, it is important for the outputs of such algorithms to be easily comprehensible. However, current tools are not designed to explicitly foster users' learning, and thus suffer from issues such as the large size and high complexity of the extracted knowledge. Moreover, due to the lack of emphasis on learning, it has previously not been studied how different ways of presenting information or interacting with data influence designers' learning. This thesis proposes new knowledge discovery methods that seek to explicitly enhance designers' learning. The methods seek to improve learning by allowing designers to be more involved in the knowledge discovery process, thus giving more control over the structure and content of the knowledge to be extracted. This makes it possible to extract design knowledge in the form that is more comprehensible and useful to the designers, and reduce the cognitive load needed to process the knowledge. This thesis presents new methods that improve four different aspects of a knowledge discovery process. First, it introduces a new data mining algorithm that utilizes domain-specific knowledge to extract generalized design knowledge. The new algorithm is based on an evolutionary algorithm equipped with special operators that generalize the knowledge encoded in features. The generalization allows extracting design rules in a more compact form compared to previous methods. Second, the thesis proposes a new framework called feature space exploration, which visually presents the extracted knowledge to designers. The framework enables designers to explicitly explore the space of possible features. This allows designers to easily compare different features in a multi-attribute space and identify the most useful ones. Third, we propose several new methods to quantitatively measure learning during design space exploration. Measuring learning is important, as it enables assessing and comparing different knowledge discovery methods on how effective they are in enhancing designers' learning. We test these measures in a human-subject experiment and examine how different ways to measure learning are related. Last, we introduce an interactive knowledge discovery method that allows designers to interactively build features that can explain the data. The new method provides interactive search capabilities that can be used to target knowledge with specific structure or content, as a designer explores the space of possible features. All proposed methods are applied to the domain of designing an Earth observing satellite system. The efficacy of the new data mining algorithm is demonstrated through a computer experiment. Other methods that directly involve human interaction are tested through human-subject experiments. The results show that the proposed methods improve users' learning over conventional approaches.
Hoffman, GuySelva Valero, Daniel
Ph. D., Mechanical Engineering
Doctor of Philosophy
dissertation or thesis