Exploration in RL

NeurIPS 2025arXiv:2504.07954

#3

Perception-R1: Pioneering Perception Policy with Reinforcement Learning

En Yu, Kangheng Lin, Liang Zhao et al.

58

ICLR 2025arXiv:2501.12735

#4

B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners

Weihao Zeng, Yuzhen Huang, Lulu Zhao et al.

Online Preference Alignment for Language Models via Count-based Exploration

Chenjia Bai, Yang Zhang, Shuang Qiu et al.

19

AAAI 2024arXiv:2308.11272

#6

MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization

Bhavya, Stelian Coros, Andreas Krause et al.

FoX: Formation-Aware Exploration in Multi-Agent Reinforcement Learning

Yonghyeon Jo, Sunwoo Lee, Junghyuk Yum et al.

multi-agent reinforcement learningpartial observabilityexploration space scalabilityformation-based equivalence+4

14

AAAI 2024arXiv:2312.12145

#8

OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments

Jinyi Liu, Zhi Wang, Yan Zheng et al.

reinforcement learningoptimistic explorationcontinuous controlenvironmental stochasticity+3

13

CVPR 2025arXiv:2503.17709

#9

Open-World Reinforcement Learning over Long Short-Term Imagination

Jiajian Li, Qi Wang, Yunbo Wang et al.

GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration

Yuchen Sun, Shanhui Zhao, Tao Yu et al.

gui agent generalizationcross-application transferabilitystructural variation modelingexploration-and-reasoning framework+3

10

ICLR 2025arXiv:2408.05804

#11

A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals

Grace Liu, Michael Tang, Benjamin Eysenbach

contrastive reinforcement learningskill emergencedirected explorationreward-free learning+2

9

ICLR 2024arXiv:2305.01521

#12

Unlocking the Power of Representations in Long-term Novelty-based Exploration

Alaa Saade, Steven Kapturowski, Daniele Calandriello et al.

9

ICLR 2025arXiv:2406.01309

#13

REvolve: Reward Evolution with Large Language Models using Human Feedback

RISHI HAZRA, Alkis Sygkounas, Andreas Persson et al.

reward function designreinforcement learninglarge language modelshuman feedback integration+3

8

AAAI 2024arXiv:2402.02097

#14

Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing

Haobin Jiang, Ziluo Ding, Zongqing Lu

multi-agent reinforcement learningcoordinated explorationnovelty sharingdecentralized cooperation+4

8

ICML 2025arXiv:2410.18076

#15

Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration

Max Wilcoxson, Qiyang Li, Kevin Frans et al.

7

ICML 2025arXiv:2503.01584

#16

Episodic Novelty Through Temporal Distance

Yuhua Jiang, Qihan Liu, Yiqin Yang et al.

UTILITY: Utilizing Explainable Reinforcement Learning to Improve Reinforcement Learning

Shicheng Liu, Minghui Zhu

SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

Cansu Sancaktar, Christian Gumbsch, Andrii Zadaianchuk et al.

4

ICML 2025arXiv:2502.07709

#19

MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces

Loris Gaven, Thomas Carta, Clément Romac et al.

4

NeurIPS 2025arXiv:2505.15293

#20

ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learning

Ruiyang Zhou, Shuozhe Li, Amy Zhang et al.

LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models

Qianyue Hao, Yiwen Song, Qingmin Liao et al.

reinforcement learningpolicy explorationlarge language modelsstochastic processes+4

NeurIPS 2025arXiv:2412.05718

#22

RLZero: Direct Policy Inference from Language Without In-Domain Supervision

Harshit Sushil Sikchi, Siddhant Agarwal, Pranaya Jajoo et al.

language-conditioned reinforcement learningzero-shot policy inferenceunsupervised rl pretrainingvideo generative models+3

ICLR 2025arXiv:2410.04988

#23

Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling

Jasmine Bayrooti, Carl Ek, Amanda Prorok

model-based reinforcement learningoptimistic explorationthompson samplingcontinuous control tasks+4

NeurIPS 2025arXiv:2506.06981

#24

Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments

Riley Simmons-Edler, Ryan Badman, Felix Berg et al.

ICML 2025arXiv:2412.16318

#25

Principal-Agent Bandit Games with Self-Interested and Exploratory Learning Agents

Junyan Liu, Lillian Ratliff

ICLR 2024arXiv:2403.01636

#26

Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks

Ziping Xu, Zifan Xu, Runxuan Jiang et al.

2

ICML 2025arXiv:2505.12204

#27

Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agents

Shuo Han, German Espinosa, Junda Huang et al.

2

ICLR 2025arXiv:2410.15184

#28

Action abstractions for amortized sampling

Oussama Boussif, Léna Ezzine, Joseph Viviano et al.

2

ICLR 2025arXiv:2502.20341

#29

Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration

Heyang Zhao, Xingrui Yu, David Bossens et al.

Safety Representations for Safer Policy Learning

Kaustubh Mani, Vincent Mai, Charlie Gauthier et al.

safety representationssafe policy learningsafe explorationreinforcement learning+2

NeurIPS 2025arXiv:2506.00138

#31

Minimax Optimal Reinforcement Learning with Quasi-Optimism

Harin Lee, Min-hwan Oh

Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics

Reece Keller, Alyn Kirsch, Felix Pei et al.

model-based intrinsic motivationautonomous explorationwhole-brain dynamicsneural-glial computation+4

ICML 2025arXiv:2507.09041

#33

Behavioral Exploration: Learning to Explore via In-Context Adaptation

Andrew Wagenmaker, Zhiyuan Zhou, Sergey Levine

ICML 2025arXiv:2503.18234

#34

KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies

Shih-Min Yang, Martin Magnusson, Johannes Stork et al.

ICML 2025arXiv:2505.12611

#35

Action-Dependent Optimality-Preserving Reward Shaping

Grant Forbes, Jianxun Wang, Leonardo Villalobos-Arias et al.

CVPR 2025arXiv:2506.09473

#36

Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learning

Cheng Chen, Yunpeng Zhai, Yifan Zhao et al.

AAAI 2024arXiv:2307.05209

#37

Abstract and Explore: A Novel Behavioral Metric with Cyclic Dynamics in Reinforcement Learning

Anjie Zhu, Peng-Fei Zhang, Ruihong Qiu et al.

Contextual Pre-planning on Reward Machine Abstractions for Enhanced Transfer in Deep Reinforcement Learning

Guy Azran, Mohamad H Danesh, Stefano Albrecht et al.

reward machinesdeep reinforcement learningtransfer learningsymbolic representations+3

AAAI 2024arXiv:2402.17978

#39

Flat Reward in Policy Parameter Space Implies Robust Reinforcement Learning

HyunKyu Lee, Sung Whan Yoon

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

Zeyang Liu, Lipeng Wan, Xinrui Yang et al.

multi-agent reinforcement learningexploration methodsintrinsic rewardsrole-based learning+4

AAAI 2024arXiv:2305.11476

#41

Forced Exploration in Bandit Problems

Qi Han, Li Zhu, Fei Guo

Exploring Plausible Preference of Body-centric Locomotion with Reinforcement Learning in Virtual Reality

Zijun Mai, BoYu Gao, Huawei Tu et al.

CrystalBox: Future-Based Explanations for Input-Driven Deep RL Systems

Learning Diverse Risk Preferences in Population-Based Self-Play

Yuhua Jiang, Qihan Liu, Xiaoteng Ma et al.

reinforcement learningself-play algorithmsrisk preferencespolicy diversity+4

multi-agent reinforcement learningdecentralized policiestrajectory entropy maximizationcontrastive trajectory representation+4

#45

Toward Efficient Multi-Agent Exploration With Trajectory Entropy Maximization

Tianxu Li, Kun Zhu

ICLR 2025

ICLR 2025arXiv:2410.09486

#46

Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning

Yantian Zha, Lin Guan, Subbarao Kambhampati

Two-Stage Evolutionary Reinforcement Learning for Enhancing Exploration and Exploitation

AIR: Unifying Individual and Collective Exploration in Cooperative Multi-Agent Reinforcement Learning

Guangchong Zhou, Zeren Zhang, Guoliang Fan

ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning

Yarden As, Bhavya, Lenart Treven et al.

reinforcement learningsafe explorationmodel-based rlprobabilistic dynamics model+3

NeurIPS 2025arXiv:2506.01685

#50

Exploration-Driven Generative Interactive Environments

Nedko Savov, Naser Kazemi, Mohammad Mahdi et al.

Cognitive Predictive Processing: A Human-inspired Framework for Adaptive Exploration in Open-World Reinforcement Learning

boheng liu, Ziyu Li, Chenghua Duan et al.

DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning

Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter et al.

Geometry Meets Incentives: Sample-Efficient Incentivized Exploration with Linear Contexts

Ben Schiffer, Mark Sellke

incentivized explorationlinear banditssample complexitycontextual bandits+4

#54

Trajectory Graph Learning: Aligning with Long Trajectories in Reinforcement Learning Without Reward Design

Yunfan Li, Eric Liu, Lin Yang

trajectory alignmentreward function designinverse reinforcement learningpreference-based rl+4

NeurIPS 2025arXiv:2510.25529

#55

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF

Tengyang Xie, Dylan Foster, Akshay Krishnamurthy et al.

Off-policy Reinforcement Learning with Model-based Exploration Augmentation

Likun Wang, Xiangteng Zhang, Yinuo Wang et al.

off-policy reinforcement learningmodel-based explorationdiffusion-based generatorcritical state synthesis+4

ICLR 2025arXiv:2502.10097

#57

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

Jonathan Light, Min Cai, Weiqin Chen et al.

Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction

Anthony GX-Chen, Kenneth Marino, Rob Fergus

Causal Information Prioritization for Efficient Reinforcement Learning

Hongye Cao, Fan Feng, Tianpei Yang et al.

causal reinforcement learningsample efficiencyfactored mdpscausal information prioritization+3

NeurIPS 2025arXiv:2506.16396

#60

Counteractive RL: Rethinking Core Principles for Efficient and Scalable Deep Reinforcement Learning

Ezgi Korkmaz

GoalLadder: Incremental Goal Discovery with Vision-Language Models

Alexey Zakharov, Shimon Whiteson

vision-language modelsreinforcement learningreward shapinggoal discovery+4

NeurIPS 2025arXiv:2506.08745

#62

Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning

Kongcheng Zhang, QI YAO, Shunyu Liu et al.

self-rewarding reinforcement learninglarge language model reasoningreasoning trajectory consistencyintrinsic reward mechanism+4

#63

Infrequent Exploration in Linear Bandits

Harin Lee, Min-hwan Oh

R1-ShareVL: Incentivizing Reasoning Capabilities of Multimodal Large Language Models via Share-GRPO

Huanjin Yao, Qixiang Yin, Jingyi Zhang et al.

Agentic RL Scaling Law: Spontaneous Code Execution for Mathematical Problem Solving

Xinji Mai, Haotian Xu, Xing W et al.

reinforcement learning scalingtool-integrated reasoningspontaneous code executionmathematical problem solving+4

NeurIPS 2025arXiv:2506.04723

#66

1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities

Kevin Wang, Ishaan Javali, Michał Bortkiewicz et al.

LaRes: Evolutionary Reinforcement Learning with LLM-based Adaptive Reward Search

Pengyi Li, Hongyao Tang, Jinbin Qiao et al.

Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning

Jiayu Wang, Yifei Ming, Zixuan Ke et al.

reinforcement learning tuningmathematical reasoningplan following executionknowledge integration+4

NeurIPS 2025arXiv:2509.20648

#69

Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration

Yiyuan Pan, Zhe Liu, Hesheng Wang

multi-agent reinforcement learningintrinsic motivationartificial curiositysparse rewards+4

#70

Deployment Efficient Reward-Free Exploration with Linear Function Approximation

Zihan Zhang, Yuxin Chen, Jason Lee et al.

reward-free explorationlinear function approximationdeployment efficiencylinear markov decision process+4

NeurIPS 2025arXiv:2501.18009

#71

Large Language Models Think Too Fast To Explore Effectively

Lan Pan, Hanbo Xie, Robert Wilson

large language modelsexploration capacityopen-ended tasksuncertainty-driven strategies+4

NeurIPS 2025arXiv:2504.03206

#72

Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward

Yanming Wan, Jiaxing Wu, Marwa Abdulhai et al.

personalized dialogue systemsmulti-turn reinforcement learningcuriosity reward mechanismuser modeling+4

NeurIPS 2025arXiv:2506.09508

#73

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL

Joey Hong, Anca Dragan, Sergey Levine

Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design

Andreas Schlaginhaufen, Reda Ouhamma, Maryam Kamgarpour

preference-based reinforcement learninghuman feedback learningmarkov decision processesrandomized exploration+4

#75

Avoiding exp(R) scaling in RLHF through Preference-based Exploration

Mingyu Chen, Yiding Chen, Wen Sun et al.

reinforcement learning from human feedbackonline rlhfsample efficiencypreference-based exploration+3

NeurIPS 2025arXiv:2512.02423

#76

Improving Reward Models with Proximal Policy Exploration for Preference-Based Reinforcement Learning

Yiwen Zhu, Jinyi Liu, Pengjie Gu et al.

EVAAA: A Virtual Environment Platform for Essential Variables in Autonomous and Adaptive Agents

Sungwoo Lee, Jungmin Lee, Sohee Kim et al.

GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning

Haolong Yan, Yeqing Shen, Xin Huang et al.

gui agent navigationscreen navigation challengessimulation environment enginemulti-turn reinforcement learning+4

#79

Uncertainty-Guided Exploration for Efficient AlphaZero Training

Scott Cheng, Meng-Yu Tsai, Ding-Yong Hong et al.

monte carlo tree searchself-play traininguncertainty-guided explorationbayesian inference framework+4

NeurIPS 2025arXiv:2506.13690

#80

Meta-learning how to Share Credit among Macro-Actions

Ionel-Alexandru Hosu, Traian Rebedea, Razvan Pascanu

macro-actionscredit assignmentexploration efficiencyreinforcement learning+3

ICLR 2025arXiv:2405.15143

#81

Exploration from a Primal-Dual Lens: Value-Incentivized Actor-Critic Methods for Sample-Efficient Online RL

Tong Yang, Bo Dai, Lin Xiao et al.

Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models

Cong Lu, Shengran Hu, Jeff Clune

hard-exploration problemsfoundation modelsrobotic controlreinforcement learning+4

ICML 2025arXiv:2412.03767

#83

Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning

Yiran Wang, Chenshu Liu, Yunfan Li et al.

model-based reinforcement learningsample efficiencygeneralist agentsstrategic dreaming+4

#84

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

Hany Hamed, Subin Kim, Dongyeong Kim et al.

ICML 2024

ICML 2024arXiv:2403.06571

#85

Explicit Exploration for High-Welfare Equilibria in Game-Theoretic Multiagent Reinforcement Learning

Austin Nguyen, Anri Gu, Michael Wellman

Scalable Online Exploration via Coverability

Philip Amortila, Dylan Foster, Akshay Krishnamurthy

reinforcement learning explorationpolicy optimization objectivesl1-coverage objectivelow coverability mdps+4

ICLR 2024arXiv:2312.08369

#87

Diversifying Robot Locomotion Behaviors with Extrinsic Behavioral Curiosity

Zhenglin Wan, Xingrui Yu, David Bossens et al.

In-context Exploration-Exploitation for Reinforcement Learning

Zhenwen Dai, Federico Tomasi, Sina Ghiassian

The Effective Horizon Explains Deep RL Performance in Stochastic Environments

Cassidy Laidlaw, Banghua Zhu, Stuart Russell et al.

ICLR 2024arXiv:2305.18246

#90

EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration

Allen Nie, Yi Su, Bo Chang et al.

Preference Optimization for Combinatorial Optimization Problems

Mingjun Pan, Guanquan Lin, You-Wei Luo et al.

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo

Haque Ishfaq, Qingfeng Lan, Pan Xu et al.

ICML 2025arXiv:2411.18825

#93

Strategic Planning: A Top-Down Approach to Option Generation

Max Ruiz Luyten, Antonin Berthon, Mihaela van der Schaar

ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Robotics

Letian Chen, Nina Moorman, Matthew Gombolay

ICLR 2024arXiv:2310.18144

#95

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

Guowei Xu, Ruijie Zheng, Yongyuan Liang et al.

Replay across Experiments: A Natural Extension of Off-Policy RL

Dhruva Tirumala, Thomas Lampe, Jose Enrique Chen et al.

Improving Intrinsic Exploration by Creating Stationary Objectives

Roger Creus Castanyer, Joshua Romoff, Glen Berseth

ICML 2025arXiv:2601.11953

#98

Controlling Underestimation Bias in Constrained Reinforcement Learning for Safe Exploration

Shiqing Gao, Jiaxin Ding, Luoyi Fu et al.

reinforcement learningsparse rewardsexplanation methodsinitial state distribution+3

#99

RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation

Zelei Cheng, Xian Wu, Jiahao Yu et al.

ICML 2024

global reinforcement learningsubmodular optimizationcurvature-dependent approximationexperiment design+4

#100

Global Reinforcement Learning : Beyond Linear and Convex Rewards via Submodular Semi-gradient Methods

Riccardo De Santi, Manish Prajapat, Andreas Krause

ICML 2024