ICML "in-context learning" Papers
50 papers found
Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
Bilgehan Sel, Ahmad Al-Tawaha, Vanshaj Khattar et al.
An Information-Theoretic Analysis of In-Context Learning
Hong Jun Jeon, Jason Lee, Qi Lei et al.
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
Zhifeng Kong, ARUSHI GOEL, Rohan Badlani et al.
BAGEL: Bootstrapping Agents by Guiding Exploration with Language
Shikhar Murty, Christopher Manning, Peter Shaw et al.
Breaking through the learning plateaus of in-context learning in Transformer
Jingwen Fu, Tao Yang, Yuwang Wang et al.
Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?
Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi et al.
Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
Jong Ho Park, Jaden Park, Zheyang Xiong et al.
Compositional Text-to-Image Generation with Dense Blob Representations
Weili Nie, Sifei Liu, Morteza Mardani et al.
Dual Operating Modes of In-Context Learning
Ziqian Lin, Kangwook Lee
Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems
David T. Hoffmann, Simon Schrodi, Jelena Bratulić et al.
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian Chen, Tianyang Hu, Hui Jin et al.
Feedback Loops With Language Models Drive In-Context Reward Hacking
Alexander Pan, Erik Jones, Meena Jagadeesan et al.
FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction
Zhonghang Li, Lianghao Xia, Yong Xu et al.
Fool Your (Vision and) Language Model with Embarrassingly Simple Permutations
Yongshuo Zong, Tingyang Yu, Ruchika Chavhan et al.
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
Jianliang He, Siyu Chen, Fengzhuo Zhang et al.
Generalization to New Sequential Decision Making Tasks with In-Context Learning
Sharath Chandra Raparthy, Eric Hambro, Robert Kirk et al.
GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks
Shivanshu Gupta, Clemens Rosenbaum, Ethan R. Elenberg
How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?
Hongkang Li, Meng Wang, Songtao Lu et al.
How do Transformers Perform In-Context Autoregressive Learning ?
Michael Sander, Raja Giryes, Taiji Suzuki et al.
How Transformers Learn Causal Structure with Gradient Descent
Eshaan Nichani, Alex Damian, Jason Lee
In-context Convergence of Transformers
Yu Huang, Yuan Cheng, Yingbin LIANG
In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
sili huang, Jifeng Hu, Hechang Chen et al.
In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
Herilalaina Rakotoarison, Steven Adriaensen, Neeratyoy Mallik et al.
In-Context Language Learning: Architectures and Algorithms
Ekin Akyürek, Bailin Wang, Yoon Kim et al.
In-Context Learning Agents Are Asymmetric Belief Updaters
Johannes A. Schubert, Akshay Kumar Jagadish, Marcel Binz et al.
In-context Learning on Function Classes Unveiled for Transformers
Zhijie Wang, Bo Jiang, Shuai Li
In-Context Principle Learning from Mistakes
Tianjun Zhang, Aman Madaan, Luyu Gao et al.
In-Context Unlearning: Language Models as Few-Shot Unlearners
Martin Pawelczyk, Seth Neel, Himabindu Lakkaraju
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
Sheng Liu, Haotian Ye, Lei Xing et al.
Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective
Fabian Falck, Ziyu Wang, Christopher Holmes
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models
Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman et al.
Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning
Sungwon Han, Jinsung Yoon, Sercan Arik et al.
Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments
Antoine Dedieu, Wolfgang Lehrach, Guangyao Zhou et al.
Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning
Jiachen Li, Qiaozi Gao, Michael Johnston et al.
Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning
TengYe Xu, Zihao Li, Qinyuan Ren
One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts
Ruochen Wang, Sohyun An, Minhao Cheng et al.
PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning
Hyeong Kyu Choi, Sharon Li
Position: Do pretrained Transformers Learn In-Context by Gradient Descent?
Lingfeng Shen, Aayush Mishra, Daniel Khashabi
Position: Understanding LLMs Requires More Than Statistical Generalization
Patrik Reizinger, Szilvia Ujváry, Anna Mészáros et al.
Position: Video as the New Language for Real-World Decision Making
Sherry Yang, Jacob C Walker, Jack Parker-Holder et al.
Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents
Zhihan Liu, Hao Hu, Shenao Zhang et al.
RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models
Qi Lv, Hao Li, Xiang Deng et al.
Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills
Kolby Nottingham, Bodhisattwa Prasad Majumder, Bhavana Dalvi et al.
Subgoal-based Demonstration Learning for Formal Theorem Proving
Xueliang Zhao, Wenda Li, Lingpeng Kong
Trainable Transformer in Transformer
Abhishek Panigrahi, Sadhika Malladi, Mengzhou Xia et al.
Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
Xiang Cheng, Yuxin Chen, Suvrit Sra
Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape
Juno Kim, Taiji Suzuki
Unifying Image Processing as Visual Prompting Question Answering
Yihao Liu, Xiangyu Chen, Xianzheng Ma et al.
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
Aaditya Singh, Ted Moskovitz, Feilx Hill et al.
Why Larger Language Models Do In-context Learning Differently?
Zhenmei Shi, Junyi Wei, Zhuoyan Xu et al.