Publications

Generative Modeling and Sampling

1. 

   PreprintTILT: Model-Intrinsic Reward Alignment For Compositional Diffusion

   Debottam Dutta, Jianchong Chen, Jaehoon Hahm, and Romit Roy Choudhury

   arXiv, 2026

   PDF Website

   tl;dr: We present TILT (Test-time model-Intrinsic reward aLignment via Tilting), a training-free framework that poses composition in diffusion as an intrinsic reward alignment problem. The reward is defined purely in distribution space, thus modality agnostic: the same reward works across modalities like image and audio, obviating the need for external reward models. It yields a closed-form target via a variational approximation, realizable via standard diffusion sampling, and is interpretable, hence amenable to preference-based modifications.

2. 

   ICML WorkshopCan Intrinsic Rewards Enable Compositional Alignment in Diffusion?

   Debottam Dutta, Jaehoon Hahm, Jianchong Chen, and Romit Roy Choudhury

   ICML 2026 Workshop on Structured Probabilistic Inference & Generative Modeling (SPIGM)

   PDF

   tl;dr: External rewards used to guide text-to-image generation are not tailored to what the generator has learned. We ask: can a model that composes poorly tell apart its good and bad samples, and can that model-intrinsic signal help it avoid compositional failures?

3. 

   ICLRSteer Away From Mode Collisions: Improving Composition In Diffusion Models

   Debottam Dutta, Jianchong Chen, Rajalaxmi Rajagopalan, Yu-Lin Wei, and Romit Roy Choudhury

   ICLR, 2026

   PDF Code Website

   tl;dr: While text-to-image models can generate stunning visuals, they frequently fail at multi-concept prompts by dropping or overshadowing weaker elements. We hypothesize this happens because the joint probability distribution overlaps too heavily with single-concept distributions, pulling generation toward a dominant concept. To resolve this without retraining, our Concept-Contrasting Corrector (CO3) steers sampling away from overlapping areas and toward "pure" modes where all concepts achieve a balanced visual presence.

4. 

   ICMLPersonalized Image Generation via Human-in-the-loop Bayesian Optimization

   Rajalaxmi Rajagopalan,  Debottam Dutta, Yu-Lin Wei, and Romit Roy Choudhury

   ICML, 2026

   PDF Website

   tl;dr: MultiBO enables precise image personalization via human-in-the-loop Bayesian optimization. It narrows the generation gap by observing that even when language-based prompting reaches its limits, humans can still visually identify if a new image x^+ is closer to their imagined target x^* than previous attempts. Iterative multi-choice user feedback is then used to guide the diffusion model to the exact desired image without retraining.

5. 

   PreprintLearning Energy-based Variational Latent Prior for VAEs

   Debottam Dutta, Chaitanya Amballa, Zhongweiyang Xu, Yu-Lin Wei, and Romit Roy Choudhury

   arXiv, 2025

   PDF

   tl;dr: To tackle the fundamental "prior hole" problem that severely degrades VAE generation quality, we introduce EVaLP, a flexible energy-based prior designed to tightly align with the aggregate posterior. Our key insight is utilizing a variational sampler network to approximate the log-normalization constant, which effectively bypasses computationally expensive MCMC sampling methods. By successfully bridging this prior-posterior mismatch, our approach enables both stable model training and fast, high-quality sample generation.

6. 

   Neurips WorkshopMulti-Source Music Generation with Latent Diffusion

   Zhongweiyang Xu,  Debottam Dutta, Yu-Lin Wei, and Romit Roy Choudhury

   Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation

   PDF Demo

   tl;dr: We introduce the Multi-Source Latent Diffusion Model (MSLDM) to resolve the noisy artifacts and poor melodies of waveform-level models by compressing individual instrumental sources into distinct VAE latents. Our key insight is that training a diffusion model on these concatenated "source latents" captures inter-source harmony significantly better than modeling whole music mixtures, leveraging this compression and noise-robustness to enable highly flexible, high-quality total and partial generation of mutually coherent tracks.

7. 

   ICASSPEstimating Multi-chirp Parameters using Curvature-guided Langevin Monte Carlo

   Sattwik Basu,  Debottam Dutta, Yu-Lin Wei, and Romit Roy Choudhury

   ICASSP, 2025

   PDF

   tl;dr: Estimating higher-order multi-chirp parameters in low signal-to-noise environments is a challenging non-convex optimization problem where standard samplers frequently fail to reliably converge. To address this, we proposed a Curvature-guided Langevin Monte Carlo (CG-LMC) algorithm that adaptively tunes Gaussian smoothing using the objective function’s average curvature to reliably reach the optimal solution.

Speech/Audio Processing and Digital Health