| Name | Yiming Yang |
| Contact Info | yiming at cs dot cmu dot edu |
| Office Location | GHC 5703, 5000 Forbes Avenue, Pittsburgh, PA 15213 |
| Office Hours | TBD |
| TA Name | TBD |
| TA Contact Info | TBD |
| Office Location | TBD |
| Office Hours | TBD |
What is this course about?
Modern machine learning is no longer dominated by a single generative paradigm. In addition to autoregressive language models, researchers increasingly use diffusion models, flow matching, energy-based methods, and consistency-style models across many frontier problems.
This course studies six major modern generative paradigms and systematically compares their strengths and limitations across three broad categories of data: token sequences, including language; continuous signals, such as images; and graph-based structures.
Why this course?
Large language models are the most visible success of modern generative AI, but they are not the whole story. Cutting-edge ML research increasingly studies generation for images, structured objects, scientific discovery, and hard optimization problems.
The course is designed for students who want to understand what lies beyond autoregressive LLMs: how different generative paradigms work, where they are being applied, and what current research questions remain open.
Who should take this course?
- NLP and LLM students interested in diffusion, flow matching, and alternatives to autoregressive models.
- ML students interested in modern generative modeling across multiple data types and applications.
- CSD students interested in learning-based approaches to NP-hard problems and graph-structured generation.
- OR/Economics students interested in machine-learning-based optimization, decision making, and structured generative methods.
Prerequisites
Students should have prior exposure to introductory machine learning, probability and statistics, and basic optimization. Familiarity with modern large language models is strongly recommended but not required. Prior experience with diffusion models, flow matching, graph generation, combinatorial optimization, or advanced deep learning systems is not assumed.
Course organization
Instructor-led lectures
The instructor will introduce each core generative paradigm through canonical examples.
- Autoregressive models through language modeling
- Energy-based models through image generation
- Diffusion models through image generation
- Flow matching through image generation
- Consistency / IMM-style models through image generation
- Autoregressive reinforcement learning for combinatorial optimization
- Non-autoregressive reinforcement learning for combinatorial optimization
- Diffusion models for combinatorial optimization
Student-led lectures
Student teams will present frontier research topics. Each topic takes one lecture, covers at least two generative paradigms, and is based on a curated paper set.
| Object Category | Possible Topics |
|---|---|
| Sequential Tokens | Coding, mathematical reasoning, diffusion language models |
| Continuous Signals | Fast generation, controllable generation, diffusion vs. flow matching |
| Graph Structures | MIS, MaxCut, SAT, molecules, proteins, other NP-hard problems |
Course format
- No traditional homework.
- No midterm or final exams.
- Each lecture includes one to three checkpoint questions, taking approximately 10–15 minutes.
- Students will submit answers to checkpoint questions through Gradescope. AI tools may not be used before submission.
- Each student will give a student-led lecture. Teaming is possible, depending on the number of enrolled students.
See the following pages for guidelines, topics, evaluation rubrics, the checkpoint grading rubric, and the course schedule.
Lecture slides: Instructor, invited, and student-led lecture slides are class-member-only materials and will be distributed to enrolled students through the course OneDrive folder Shared_slides. They are not posted on this public website.
Grading
| Component | Weight |
|---|---|
| Instructor-led Lecture Checkpoints | 30% |
| Student-led Lecture Checkpoints | 30% |
| Student-Led Lectures | 40% (50% by peers, 50% by Instructor/TA) |
Course Schedule
The first part of the semester consists of instructor-led lectures and three invited student-led exemplars. After Spring Break, the remaining lecture slots are reserved for student-led lectures; their topics and presenters will be determined after enrollment and topic selection.
| Date | ID | Lecture | Planning Focus / Notes |
|---|---|---|---|
| 1/19 | L1 | Course Introduction | What do we cover and how do we run the course |
| 1/21 | L2 | Autoregressive Modeling Foundations | |
| 1/26 | L3 | Energy-Based Modeling Fundamentals | |
| 1/28 | L4 | Diffusion Models Fundamentals I | From Energy Landscapes to Hierarchical Denoising |
| 2/2 | L5 | Diffusion Models Fundamentals II | Diffusion as Score-Based Modeling |
| 2/4 | L6 | Flow Matching Fundamentals | Learning Arbitrary Distribution Transformations |
| 2/9 | L7 | Inductive Moment Matching (IMM) | Learning Consistent Distribution Transformations |
| 2/11 | L8 | Autoregressive Reinforcement Learning for Neural Combinatorial Optimization (AM) | Attention Model (AM), Kool et al., ICLR 2019 |
| 2/16 | L9 | Non-Autoregressive Reinforcement Learning for Neural Combinatorial Optimization | DIMES, Qiu et al., NeurIPS 2022 |
| 2/18 | L10 | Diffusion Models for Combinatorial Optimization | Supervised DIFUSCO (Sun & Yang, NeurIPS 2023) and Unsupervised DiffUCO (Sanokowski et al., ICML 2024) |
| 2/23 | L11 | What Makes Diffusion Work for Combinatorial Optimization | Combinatorial Adjoint Matching (CAM), Feng et al., ICML 2026 |
| 2/25 | Guest Lecture | Discrete Diffusion for Complex Reasoning, Planning and Beyond | by Jiacheng Ye, UHK |
| 3/2 | SE1 | Mathematical Reasoning & Proof Generation | Invited student-led lecture exemplar I by Weihua Du |
| 3/4 | SE2 | Molecular Graph Generation | Invited student-led lecture exemplar II by Shanda Li |
| 3/9 | — | Spring Break | No class |
| 3/11 | — | Spring Break | No class |
| 3/16 | SE3 | Node Selection from Graphs in Combinatorial Optimization | Invited student-led lecture exemplar III by Shengyu Feng |
| 3/18 | SL2 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 3/23 | SL3 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 3/25 | SL4 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 3/30 | SL5 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/1 | SL6 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/6 | SL7 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/8 | SL8 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/13 | SL9 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/15–4/16 | — | Spring Carnival | No class |
| 4/20 | SL10 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/22 | SL11 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/27 | SL12 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |
| 4/29 | SL13 | Student-led lecture | Topic and presenter(s) to be determined after enrollment and topic selection |