Large-scale generative models, which can be steered to optimize specific objectives, have yielded remarkable successes in scientific applications such as de-novo protein design.  However, a central challenge in scientific discovery is to explore beyond the domain well-represented by the training data.  
In this talk, I will present recent work leveraging ideas from reinforcement learning and stochastic optimal control to steer generative models for novelty-seeking generative discovery.  In particular, I will introduce Flow Density Control, a flexible framework for steering flow- and diffusion-based generative models that captures diverse use-cases, including maximum entropy manifold exploration and tail-aware generative optimization.  I will also discuss how verifiers (assessing, e.g., physical plausibility) can be utilized to constrain and guide the exploration process.  I will motivate and illustrate the approaches on several examples from molecular design.