Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell University Graduate School
  3. Cornell Theses and Dissertations
  4. KNOWLEDGE DISTILLATION FOR MOLECULAR PROPERTY PREDICTION

KNOWLEDGE DISTILLATION FOR MOLECULAR PROPERTY PREDICTION

File(s)
Sheshanarayana_cornell_0058O_12347.pdf (5.75 MB)
Permanent Link(s)
https://doi.org/10.7298/rtyv-3e85
https://hdl.handle.net/1813/120981
Collections
Cornell Theses and Dissertations
Author
Sheshanarayana, Rahul
Abstract

Knowledge distillation (KD) is a powerful model compression technique that transfers knowledge from complex teacher models to compact student models, reducing computational costs while preserving predictive accuracy. This study investigated KD's efficacy in molecular property prediction across domain-specific and cross-domain tasks, leveraging state-of-the-art graph neural networks (SchNet, DimeNet++, and TensorNet). In the domain-specific setting, KD improved regression performance across diverse quantum mechanical properties in the QM9 dataset, with DimeNet++ student models achieving up to a 90% improvement in R^2 compared to non-KD baselines. Notably, in certain cases, smaller student models achieved comparable or even superior R^2 improvements while being 2× smaller, highlighting KD’s ability to enhance efficiency without sacrificing predictive performance. Cross-domain evaluations further demonstrated KD’s adaptability, where embeddings from QM9-trained teacher models enhanced predictions for ESOL (water solubility of molecules) and FreeSolv (hydration free energy of molecules), with SchNet exhibiting the highest gains of approximately 65% in water solubility predictions. Embedding analysis revealed substantial student-teacher alignment gains, with the relative shift in cosine similarity distribution peaks reaching up to 1.0 across student models. These findings highlighted KD as a robust strategy for enhancing molecular representation learning, with implications for cheminformatics, materials science, and drug discovery.

Description
45 pages
Date Issued
2025-12
Keywords
Graph neural networks
•
Knowledge distillation
•
Machine learning
•
Scalability
Committee Chair
You, Fengqi
Committee Member
Acharya, Jayadev
Degree Discipline
Systems Engineering
Degree Name
M.S., Systems Engineering
Degree Level
Master of Science
Type
dissertation or thesis

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance