https://astro-eric.github.io/Recent content on Haozhe JiangHugoen-usThu, 11 Jun 2026 00:00:00 -0700https://astro-eric.github.io/blogs/dipod/Thu, 11 Jun 2026 00:00:00 -0700https://astro-eric.github.io/blogs/dipod/<h1 id="dipod-diffusion-policy-optimization-without-drifting-apart">DiPOD: Diffusion Policy Optimization without Drifting Apart</h1> <p>Diffusion language models are an exciting alternative to autoregressive language models. Instead of generating a response strictly left-to-right, they can refine many tokens in parallel. This opens the door to faster sampling, flexible decoding orders, and possibly new ways of doing long-form reasoning.</p> <p>But there is a gap between this promise and the current reality. Reinforcement learning (RL), especially policy-gradient methods, has become one of the main reasons modern autoregressive LLMs got much better at math, code, and instruction following. For diffusion language models, however, RL post-training has not paid off in the same way. Training is often unstable, reward curves wobble, and reasoning performance still lags behind autoregressive counterparts.</p>https://astro-eric.github.io/family-tree/Thu, 11 Jun 2026 00:00:00 -0700https://astro-eric.github.io/family-tree/https://astro-eric.github.io/gallery/beijing/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/beijing/https://astro-eric.github.io/gallery/california/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/california/https://astro-eric.github.io/blogs/surjective/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/blogs/surjective/<h1 id="large-language-models-are-surjective-injective-invertible">Large Language Models are Surjective? Injective? Invertible?</h1> <p>Recently, there have been discussions on functional properties of Transformers, the basic building block of Large Language Models (LLM) and many other generative models. <a href="https://arxiv.org/abs/2508.19445">My paper</a> ([1]) proves that Transformers can output anything given an appropriate input (surjective). After a few months, <a href="https://arxiv.org/abs/2510.15511">a followup</a> ([2]) proves that LLMs always send different inputs to different outputs (injective), and hence we can invert the outputs back to inputs. With these claims in mind, wild thoughts about generative models start flowing around. Are jailbreaks fundamentally unavoidable? Are Transformers lossless compression of knowledge? Do we have no privacy when we use LLMs? Does combining the two papers imply that LLMs are bijective? In this blog, we clarify the concepts of surjectivity, injectivity and invertibility that appear in these papers and explain their true implications.</p>https://astro-eric.github.io/gallery/hangzhou/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/hangzhou/https://astro-eric.github.io/gallery/hawaii/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/hawaii/https://astro-eric.github.io/gallery/princeton/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/princeton/https://astro-eric.github.io/gallery/rwanda/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/rwanda/https://astro-eric.github.io/gallery/seattle/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/seattle/https://astro-eric.github.io/gallery/shanghai/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/shanghai/https://astro-eric.github.io/gallery/vienna/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/vienna/https://astro-eric.github.io/gallery/yili/Sun, 16 Nov 2025 23:29:21 +0530https://astro-eric.github.io/gallery/yili/https://astro-eric.github.io/publications/Mon, 01 Jan 0001 00:00:00 +0000https://astro-eric.github.io/publications/<div class="pub-search-index" style="display:none"> Diffusion Policy Optimization without Drifting Apart Haozhe Jiang, Haiwen Feng, Jiantao Jiao, Pieter Abbeel, Angjoo Kanazawa , Nika Haghtalab Reinforcement Learning Diffusion Model Generative Model We develop a framework to analyze policy gradient for diffusion models, while preserving the structure of diffusion models. We identify the &#39;double drift&#39; phenomenon when one replaces the log-likelihood with proxies. The ELBO would drift from log-likelihood and the gradient would drift from policy gradient. We solve it by DiPOD, which interleaves the gradients with self-distillation steps, leading to significant performance boosts in multiple domains. Diffusion Language Models are Provably Optimal Parallel Samplers Haozhe Jiang, Nika Haghtalab, Lijie Chen Theory Deep Learning Diffusion Model ICLR 2026 We prove that masked diffusion models can generate distributions with the fewest possible steps, and further fewest possible memroy with remasking/revision. We also prove that with remasking/revision, a masked diffusion model with fixed sequence length can generate more distributions with remasking/revision than without. On Surjectivity of Neural Networks: Can you elicit any behavior from your model? Haozhe Jiang, Nika Haghtalab Theory Deep Learning Generative Model We prove that a lot of practical architures for generative models are surjective, regardless of how they are trained. This means that any output, including undesirable ones, can be elicited by some input, leading to a fundamental vulnerability to jailbreaks. A Black-box Approach for Non-stationary Multi-agent Reinforcement Learning Haozhe Jiang, Qiwen Cui, Zhihan Xiong, Maryam Fazel, Simon S Du Theory Game Theory ICLR 2024 Can we track Equilibria in non-stationary Multi-agent systems? We comprehensively analyze this problem and find most non-stationary bandit algorithms fail to generalize. In multi-agent systems, **different actions compete with different best responses**. This forbids us from comparing actions by comparing their own rewards, sabotaging all test-based algorithms. We solve this problem by using an Explore-then-Commit-then-Test algorithm, in combination with prior techniques. This black-box algorithm is simple but admits a no-regret guarantee. Offline Meta Reinforcement Learning with In-Distribution Online Adaptation Jianhao Wang*, Jin Zhang*, Haozhe Jiang, Junyu Zhang, Liwei Wang, Chongjie Zhang Reinforcement Learning Meta Learning ICML 2023 Why is it hard to do offline meta-learning? We point out that the meta-learning setting induces a new type of **distribution shift between offline training and online adaptation**. We characterize this challenge and propose IDAQ, a simple greedy context-based algorithm to tackle this problem. Practically Solving LPN in High Noise Regimes Faster Using Neural Networks Haozhe Jiang*, Kaiyue Wen*, Yilei Chen Theory Cryptography Deep Learning Can we break the Learning Parity with Noise (LPN) problem with Neural Networks? Empirically, we find out that **when the noise is high, neural networks are particularly useful**. We corroborate this observation by proving that the sample complexity of some neural networks scales optimally with the noise. This is the first neural-network-based algorithm surpassing all classical counterparts in breaking cryptographic primitives. Offline congestion games: How feedback type affects data coverage requirement Haozhe Jiang*, Qiwen Cui*, Zhihan Xiong, Maryam Fazel, Simon S Du Theory Game Theory ICLR 2023 If we want to learn Nash Equilibrium in congestion games, what offline dataset should we have? By looking at **dataset coverage in facility space** instead of action space, we find out the minimal dataset coverage requirement. We also present efficient learning algorithms and demonstrate the **separation** of coverage requirements under different feedback models. Offline reinforcement learning with reverse model-based imagination Jianhao Wang*, Wenzhe Li*, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, Chongjie Zhang Reinforcement Learning NeurIPS 2021 You can’t connect the dots looking forward; you can only connect them looking backwards. How to learn RL policy generalizing from offline datasets while avoiding dangerous actions? We propose to **learn a reverse dynamic model** instead of a forward one. If a forward model goes wrong, the learned policy may take dangerous actions. If a reverse model goes wrong, the agent still acts safely because it could not start at the wrong reversed state in the first place. </div>