Chapter 6

Applications — Learning Predictive, Generative and Representation Models

Unity of knowledge and action!

In this chapter, we present applications of stochastic compositional optimization and finite-sum coupled compositional optimization (FCCO) in both supervised and self-supervised learning settings. These include training predictive models, generative models, and representation models using advanced objective functions such as distributionally robust optimization (DRO), group DRO (GDRO), AUC losses, NDCG loss, discriminative X-risks and contrastive losses. We also highlight a key application of FCCO in solving multiple inequality-constrained optimization problems.

---

Contents

• 6.1 Stochastic Optimization Framework
• 6.2 DRO and Group DRO
• 6.3 Stochastic AUC and NDCG Maximization
• 6.4 Discriminative Pretraining of Representation Models
• 6.5 Discriminative Fine-tuning of Large Language Models
• 6.6 Constrained Learning
• 6.7 Learning Data Compositional Networks
• 6.8 Model Steering by DRRHO Risk Minimization
• 6.9 Notes and Discussion