Offline Reinforcement Learning

ICLR 2025arXiv:2412.07762

#2

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data

Zhiyuan Zhou, Andy Peng, Qiyang Li et al.

reinforcement learning fine-tuningoffline reinforcement learningonline reinforcement learningdistribution mismatch+4

27

AAAI 2024arXiv:2306.12755

#3

Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning

Jinxin Liu, Ziqi Zhang, Zhenyu Wei et al.

offline reinforcement learningout-of-distribution state actionscross-domain learningtransition dynamics mismatch+4

26

AAAI 2025arXiv:2412.11120

#4

Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning

Yun Qu, Yuhang Jiang, Boyuan Wang et al.

23

ICML 2025arXiv:2505.07395

#5

ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning

Hongyin Zhang, Zifeng Zhuang, Han Zhao et al.

20

NeurIPS 2025arXiv:2505.20347

#6

Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards

Zijing Hu, Fengda Zhang, Long Chen et al.

SeRL: Self-play Reinforcement Learning for Large Language Models with Limited Data

Wenkai Fang, Shunyu Liu, Yang Zhou et al.

reinforcement learninglarge language modelsself-instruction generationself-rewarding mechanisms+4

19

ICLR 2025arXiv:2411.00418

#8

SELF-EVOLVED REWARD LEARNING FOR LLMS

Chenghua Huang, Zhizhen Fan, Lu Wang et al.

reinforcement learning from human feedbackreward model trainingself-evolved learninglanguage model alignment+3

18

AAAI 2024arXiv:2312.15909

#9

Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations

Renzhe Zhou, Chen-Xiao Gao, Zongzhang Zhang et al.

offline meta-reinforcement learningtask representation learningdata limitationstask auto-encoder+4

14

ICLR 2024arXiv:2310.05422

#10

Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning

Fan-Ming Luo, Tian Xu, Xingchen Cao et al.

14

NeurIPS 2025arXiv:2506.20520

#11

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards

Charles Arnal, Gaëtan Narozniak, Vivien Cabannes et al.

13

AAAI 2025arXiv:2501.06773

#12

XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning

Alexander Nikulin, Ilya Zisman, Alexey Zemtsov et al.

Pareto Set Learning for Multi-Objective Reinforcement Learning

Erlong Liu, Yu-Chang Wu, Xiaobin Huang et al.

10

ICLR 2024arXiv:2401.08819

#14

MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL

Claas Voelcker, Marcel Hussing, ERIC EATON et al.

Open-World Reinforcement Learning over Long Short-Term Imagination

Jiajian Li, Qi Wang, Yunbo Wang et al.

Learning from Sparse Offline Datasets via Conservative Density Estimation

Zhepeng Cen, Zuxin Liu, Zitong Wang et al.

10

NeurIPS 2025arXiv:2506.00070

#17

Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics

Dongyoung Kim, Huiwon Jang, Sumin Park et al.

reinforcement learningembodied reasoningrobot controlvision-language models+4

9

AAAI 2025arXiv:2412.07639

#18

Offline Multi-Agent Reinforcement Learning via In-Sample Sequential Policy Optimization

Zongkai Liu, Qian Lin, Chao Yu et al.

8

ICLR 2024arXiv:2404.16779

#19

DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks

Tongzhou Mu, Minghua Liu, Hao Su

7

AAAI 2025arXiv:2402.01886

#20

Inverse Reinforcement Learning by Estimating Expertise of Demonstrators

Mark Beliaev, Ramtin Pedarsani

ICLR 2025arXiv:2410.10132

#21

Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning

Hung Le, Dung Nguyen, Kien Do et al.

memory-augmented agentspartially observable environmentsreinforcement learninghadamard product+4

ICLR 2025arXiv:2411.07007

#22

Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching

Arnav Kumar Jain, Harley Wiltzer, Jesse Farebrother et al.

inverse reinforcement learningsuccessor feature matchingpolicy gradient descentstate-only imitation+4

CVPR 2025arXiv:2505.24816

#23

CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning

Jiangpeng He, Zhihao Duan, Fengqing Zhu

class-incremental learningparameter-efficient fine-tuninglow-rank adaptationdual-adapter architecture+4

AAAI 2025arXiv:2311.04830

#24

Real-Time Recurrent Reinforcement Learning

Julian Lemmel, Radu Grosu

5

ICLR 2025arXiv:2407.00699

#25

Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning

Kwanyoung Park, Youngwoon Lee

5

ICLR 2025arXiv:2502.04141

#26

Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning

Wesley Suttle, Aamodh Suresh, Carlos Nieto-Granda

behavioral entropyoffline reinforcement learningdataset generationstate space exploration+4

ECCV 2024arXiv:2407.14872

#27

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Yanting Yang, Minghao Chen, Qibo Qiu et al.

vision-language modelslanguage-conditioned rewardrobotic planningreinforcement learning+4

ICML 2025arXiv:2404.02113

#28

POCE: Primal Policy Optimization with Conservative Estimation for Multi-constraint Offline Reinforcement Learning

Jiayi Guan, Li Shen, Ao Zhou et al.

Position: Lifetime tuning is incompatible with continual reinforcement learning

Golnaz Mesbahi, Parham Mohammad Panahi, Olya Mastikhina et al.

ICLR 2025arXiv:2407.04285

#30

Visual Reinforcement Learning with Residual Action

Zhenxian Liu, Peixi Peng, Yonghong Tian

Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling

Jiawei Xu, Rui Yang, Shuang Qiu et al.

reward modelinghuman preference learningpolicy optimizationoff-distribution problem+4

#32

Uncertainty and Influence aware Reward Model Refinement for Reinforcement Learning from Human Feedback

Zexu Sun, Yiju Guo, Yankai Lin et al.

ICLR 2025

ICLR 2025arXiv:2410.05782

#33

Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards

Zhaohui JIANG, Xuening Feng, Paul Weng et al.

AAAI 2025arXiv:2503.01178

#34

Differentiable Information Enhanced Model-Based Reinforcement Learning

Xiaoyuan Zhang, Xinyan Cai, Bo Liu et al.

ICML 2025arXiv:2502.06861

#35

Design Considerations in Offline Preference-based RL

Alekh Agarwal, Christoph Dann, Teodor Vanislavov Marinov

ICML 2025arXiv:2507.02712

#36

A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control

Zilin Kang, Chenyuan Hu, Yu Luo et al.

ICML 2025arXiv:2506.13672

#37

The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning

Jiashun Liu, Johan Obando-Ceron, Pablo Samuel Castro et al.

NeurIPS 2025arXiv:2502.02869

#38

Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds

Fan Wang, Pengtao Shao, Yiming Zhang et al.

in-context reinforcement learningprocedurally generated mdpsmeta-trainingpolicy distillation+4

NeurIPS 2025arXiv:2505.19281

#39

Provably Efficient Online RLHF with One-Pass Reward Modeling

Long-Fei Li, Yu-Yang Qian, Peng Zhao et al.

A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning

Yuzheng Hu, Fan Wu, Haotian Ye et al.

data attributiononline reinforcement learningproximal policy optimizationgradient similarity+4

NeurIPS 2025arXiv:2506.06964

#41

Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization

Subhojyoti Mukherjee, Viet Lai, Raghavendra Addanki et al.

offline reinforcement learningreward-weighted fine-tuninglarge language modelsconversation optimization+2

NeurIPS 2025arXiv:2504.11453

#42

A Clean Slate for Offline Reinforcement Learning

Matthew T Jackson, Uljad Berdica, Jarek Liesen et al.

NeurIPS 2025arXiv:2505.23871

#43

ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning

Zeyuan Liu, Zhihe Yang, Jiawei Xu et al.

offline reinforcement learningdataset corruption recoverydiffusion modelsambient denoising diffusion probabilistic models+3

ICLR 2025arXiv:2406.18450

#44

Exact Policy Recovery in Offline RL with Both Heavy-Tailed Rewards and Data Corruption

Yiding Chen, Xuezhou Zhang, Qiaomin Xie et al.

Preference Elicitation for Offline Reinforcement Learning

Alizée Pace, Bernhard Schölkopf, Gunnar Ratsch et al.

ICLR 2025arXiv:2302.02182

#46

Online Reinforcement Learning in Non-Stationary Context-Driven Environments

Pouya Hamadanian, Arash Nasr-Esfahany, Malte Schwarzkopf et al.

AAAI 2025arXiv:2412.09104

#47

In-Dataset Trajectory Return Regularization for Offline Preference-based Reinforcement Learning

Songjun Tu, Jingbo Sun, Qichao Zhang et al.

AAAI 2025arXiv:2502.05555

#48

Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder

Yuhan Zhang, Guoqing Ma, Guangfu Hao et al.

AAAI 2024arXiv:2312.12191

#49

CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning

Chenyu Sun, Hangwei Qian, Chunyan Miao

offline reinforcement learningunsupervised data collectioncuriosity-driven explorationadaptive temporal distances+4

ICML 2025arXiv:2412.06877

#50

Dynamic Uncertainty Estimation for Offline Reinforcement Learning

Jiesheng Wang, Lin Li, Wei Wei et al.

Concurrent Reinforcement Learning with Aggregated States via Randomized Least Squares Value Iteration

Yan Chen, Jerry Bai, Yiteng Zhang et al.

The Synergy of LLMs & RL Unlocks Offline Learning of Generalizable Language-Conditioned Policies with Low-fidelity Data

Thomas Pouplin, Katarzyna Kobalczyk, Hao Sun et al.

ICML 2025arXiv:2505.19923

#53

Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL

Qin-Wen Luo, Ming-Kun Xie, Ye-Wen Wang et al.

ICCV 2025arXiv:2506.21037

#54

Reinforcement Learning-Guided Data Selection via Redundancy Assessment

Suorong Yang, Peijia Li, Furao Shen et al.

reinforcement learningdata selectionepsilon-sample coverdataset redundancy+2

NeurIPS 2025arXiv:2505.09496

#55

REINFORCEMENT LEARNING FOR INDIVIDUAL OPTIMAL POLICY FROM HETEROGENEOUS DATA

Rui Miao, Babak Shahbaba, Annie Qu

offline reinforcement learningheterogeneous dataindividualized policy optimizationmarkov decision processes+4

visual reinforcement learningrobust representation learninginformation bottleneck frameworkaction-value estimation+3

#56

Learning Robust Representations with Long-Term Information for Generalization in Visual Reinforcement Learning

Rui Yang, Jie Wang, Qijie Peng et al.

ICLR 2025

ICLR 2025arXiv:2407.20651

#57

Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations

Yupei Yang, Biwei Huang, Fan Feng et al.

reinforcement learning generalizationcausal representation learningenvironment space adaptationdistribution shift adaptation+4

ICLR 2025arXiv:2506.13536

#58

What Matters in Learning from Large-Scale Datasets for Robot Manipulation

Vaibhav Saxena, Matthew Bronars, Nadun Ranawaka Arachchige et al.

imitation learningrobot manipulationdataset compositionpolicy learning+4

ICLR 2025arXiv:2504.13368

#59

Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay

Yifan Sun, Jingyan Shen, Yibin Wang et al.

Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations

Yujie Zhu, Charles Hepburn, Matthew Thorpe et al.

An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learning

Haoran Xu, Shuozhe Li, Harshit Sikchi et al.

reinforcement learningoptimal discriminatorbehavior cloningvisitation distribution ratio+4

NeurIPS 2025arXiv:2510.18687

#62

Behavior Injection: Preparing Language Models for Reinforcement Learning

Zhepeng Cen, Yihang Yao, William Han et al.

Sample-Efficient Tabular Self-Play for Offline Robust Reinforcement Learning

Na Li, Zewu Zheng, Wei Ni et al.

Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach

Chenbei Lu, Zaiwei Chen, Tongxin Li et al.

reinforcement learningimperfect transition predictionsmulti-step predictionsbayesian value function+4

multi-agent reinforcement learningoffline reinforcement learningsafe reinforcement learningdecision transformer+4

#65

MOSDT: Self-Distillation-Based Decision Transformer for Multi-Agent Offline Safe Reinforcement Learning

Yuchen Xia, Yunjian Xu

NeurIPS 2025

ICLR 2025arXiv:2412.03258

#66

Learning on One Mode: Addressing Multi-modality in Offline Reinforcement Learning

Mianchu Wang, Yue Jin, Giovanni Montana

offline reinforcement learningmulti-modal action distributionsgaussian mixture modelweighted imitation learning+3

NeurIPS 2025arXiv:2506.09508

#67

TESTING STATIONARITY AND CHANGE POINT DETECTION IN REINFORCEMENT LEARNING

Mengbing Li, Chengchun Shi, Zhenke Wu et al.

Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design

Andreas Schlaginhaufen, Reda Ouhamma, Maryam Kamgarpour

preference-based reinforcement learninghuman feedback learningmarkov decision processesrandomized exploration+4

NeurIPS 2025arXiv:2509.01720

#69

Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control

Georgios Papoudakis, Thomas Coste, Jianye Hao et al.

off-policy reinforcement learningmobile app controlfoundation model fine-tuningsparse reward settings+4

NeurIPS 2025arXiv:2510.22158

#70

Robust and Scalable Autonomous Reinforcement Learning in Irreversible Environments

Sang-Hyun Lee

ACTIVE: Offline Reinforcement Learning via Adaptive Imitation and In-sample $V$-Ensemble

Tianyuan Chen, Ronglong Cai, Faguo Wu et al.

Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics

Lorenzo Magnino, Kai Shao, Zida Wu et al.

mean field gamesdeep reinforcement learningnon-stationary dynamicsfictitious play methodology+4

NeurIPS 2025arXiv:2507.11269

#73

Select before Act: Spatially Decoupled Action Repetition for Continuous Control

Buqing Nie, Yangqing Fu, Yue Gao

Turning Sand to Gold: Recycling Data to Bridge On-Policy and Off-Policy Learning via Causal Bound

Tal Fiskus, Uri Shaham

ICLR 2025arXiv:2406.11715

#75

Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection

Adyasha Maharana, Jaehong Yoon, Tianlong Chen et al.

Reinforcement Learning Teachers of Test Time Scaling

Edoardo Cetin, Tianyu Zhao, Yujin Tang

Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning

Calarina Muslimani, Matthew E Taylor

Measuring memorization in RLHF for code completion

Jamie Hayes, I Shumailov, Billy Porter et al.

rlhf alignment processtraining data memorizationdirect preference optimizationpreference learning methods+4

NeurIPS 2025arXiv:2511.02567

#79

Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning

Yixiu Mao, Yun Qu, Qi Wang et al.

offline reinforcement learningextrapolation error mitigationneighborhood constraintdistribution shift+4

ICLR 2025arXiv:2502.20630

#80

Subtask-Aware Visual Reward Learning from Segmented Demonstrations

Changyeon Kim, Minho Heo, Doohyun Lee et al.

reward learningsegmented demonstrationsdense reward functioncontrastive learning+4

NeurIPS 2025arXiv:2512.01987

#81

Confounding Robust Deep Reinforcement Learning: A Causal Approach

Mingxuan Li, Junzhe Zhang, Elias Bareinboim

Cross-Domain Offline Policy Adaptation with Optimal Transport and Dataset Constraint

Jiafei Lyu, Mengbei Yan, Zhongjian Qiao et al.

Forecasting in Offline Reinforcement Learning for Non-stationary Environments

Suzan Ece Ada, Georg Martius, Emre Ugur et al.

offline reinforcement learningnon-stationary environmentsconditional diffusion modelszero-shot forecasting+3

NeurIPS 2025arXiv:2502.14819

#84

Learning Preferences without Interaction for Cooperative AI: A Hybrid Offline-Online Approach

Haitong Ma, Haoran Yu, Haobo Fu et al.

Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models

Uladzislau Sobal, Wancong Zhang, Kyunghyun Cho et al.

reward-free offline learninglatent dynamics modelsmodel-based planninggoal-conditioned rl+4

NeurIPS 2025arXiv:2510.19314

#86

Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback

Wei Shen, Guanlin Liu, Yu Yue et al.

ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization

The Viet Bui, Thanh Nguyen, Tien Mai

Staggered Environment Resets Improve Massively Parallel On-Policy Reinforcement Learning

Sid Bharthulwar, Stone Tao, Hao Su

Continual Knowledge Adaptation for Reinforcement Learning

Jinwu Hu, ZiHao Lian, Zhiquan Wen et al.

reinforcement learningcontinual learningcatastrophic forgettingknowledge transfer+3

NeurIPS 2025arXiv:2511.10648

#90

Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling

Jiahao Wang, Weiye Xu, Aijun Yang et al.

outcome-reward reinforcement learningmultimodal large language modelsmultimodal reasoning benchmarksself-consistency sampling+4

NeurIPS 2025arXiv:2505.05701

#91

FANS: A Flatness-Aware Network Structure for Generalization in Offline Reinforcement Learning

Da Wang, Yi Ma, Ting Guo et al.

Pretraining a Shared Q-Network for Data-Efficient Offline Reinforcement Learning

Jongchan Park, Mingyu Park, Donghwan Lee

offline reinforcement learningq-network initializationdata efficiencyshared q-network+4

offline reinforcement learningmodel-free algorithmsrobustness enhancementdouble-pessimism principle+3

#93

Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Models

Wen-Tse Chen, Jiayu Chen, Fahim Tajwar et al.

Model-Free Offline Reinforcement Learning with Enhanced Robustness

Chi Zhang, Zain Ulabedeen Farhat, George Atia et al.

ICLR 2025

ICML 2024arXiv:2403.00514

#95

Online Pre-Training for Offline-to-Online Reinforcement Learning

Yongjae Shin, Jeonghye Kim, Whiyoung Jung et al.

Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning

Michal Nauman, Michał Bortkiewicz, Piotr Milos et al.

off-policy reinforcement learningactor-critic methodsregularization techniquesoverestimation bias+3

ICML 2025arXiv:2405.17618

#97

Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales

Ju-Seung Byun, Andrew Perrault

ICLR 2024arXiv:2402.12673

#98

Behavior-agnostic Task Inference for Robust Offline In-context Reinforcement Learning

Long Ma, Fangwei Zhong, Yizhou Wang

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

Xiangyu Liu, Chenghao Deng, Yanchao Sun et al.

ICML 2025arXiv:2505.06482

#100

Video-Enhanced Offline Reinforcement Learning: A Model-Based Approach

Minting Pan, Yitao Zheng, Jiajian Li et al.