Cornell University
Library
Cornell UniversityLibrary

eCommons

Help
Log In(current)
  1. Home
  2. Cornell University Graduate School
  3. Cornell Theses and Dissertations
  4. Driving Image Generation Using Interactive Preference Optimization

Driving Image Generation Using Interactive Preference Optimization

File(s)
Aggarwal_cornell_0058O_12125.pdf (16.6 MB)
Permanent Link(s)
https://doi.org/10.7298/7h0h-p982
https://hdl.handle.net/1813/115794
Collections
Cornell Theses and Dissertations
Author
Aggarwal, Madhav
Abstract

Humans care about working with content based on specific subjective criteria.These criteria can be challenging to evaluate and weigh appropriately, as they vary from person to person. This work explores the design of interactive tools to help users quantify subjective dimensions of data that can be presented visually. In a scenario where absolute values for subjective criteria are unavailable, the ideal way to evaluate samples is to receive constant feedback from the user. Additional nuance can be involved in defining these criteria when the problem is high dimensional. This work presents three interactive applications that leverage the strength of humans in making pairwise comparisons to develop and communicate a deeper understanding of their data. We present an interactive tool designed for the problem defined above. This tool is closely integrated within a programming environment to enable users to replicate the mental model of their subjective preferences and handle complex data effortlessly. Results from our first application demonstrate the effectiveness of our tool when ranking individual samples from extensive image collections. We are further able to simplify the process for users in crowd-sourced settings. The second application enables the discovery of novel trends given labeled collections. We test the performance on a dataset of over 50,000 images to build timelapse videos. We conclude by showcasing our third application, which performs optimization in the background for controlling the generation of new samples based on subjective criteria. Our evaluation task involves image generation using generative Machine Learning models. We specifically use Diffusion models, which have made it easy to sample high-fidelity images conditioned on simple text prompts. However, achieving precise control over the generation process using these models remains a complex problem since the criteria fed as input using text prompts can be balanced in many ways. To navigate the space of alternatives faster, we optimize utilizing the user’s feedback to construct the desired image. Based on Bayesian Optimization, our approach learns the hidden objective function represented by the user’s preferences. In summary, we present three applications that pave the way for future work in building better interfaces closely aligned with machine learning techniques.

Description
50 pages
Date Issued
2024-05
Keywords
Bayesian Optimization
•
Computer Vision
•
Generative AI
•
Human Computer Interaction
•
Machine Learning
•
Preference Optimization
Committee Chair
Davis, Myers
Committee Member
Hariharan, Bharath
Manzoor, Emaad
Degree Discipline
Computer Science
Degree Name
M.S., Computer Science
Degree Level
Master of Science
Rights
Attribution-ShareAlike 4.0 International
Rights URI
https://creativecommons.org/licenses/by-sa/4.0/
Type
dissertation or thesis
Link(s) to Catalog Record
https://newcatalog.library.cornell.edu/catalog/16575449

Site Statistics | Help

About eCommons | Policies | Terms of use | Contact Us

copyright © 2002-2026 Cornell University Library | Privacy | Web Accessibility Assistance