Song-Chun Zhu

Papers

151

Total Citations

Papers (81)

Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis

CVPR 2025

citations

Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World

ICLR 2024

citations

Neural-Symbolic Recursive Machine for Systematic Generalization

ICLR 2024

citations

Differentiable Information Enhanced Model-Based Reinforcement Learning

AAAI 2025

citations

Joint Action Recognition and Pose Estimation From Video

CVPR 2015

citations

Recognizing Car Fluents From Video

CVPR 2016

citations

Inferring Forces and Learning Human Utilities From Videos

CVPR 2016

citations

Multi-View People Tracking via Hierarchical Trajectory Composition

CVPR 2016

citations

Mining Object Parts From CNNs via Active Question-Answering

CVPR 2017arXiv

citations

CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

CVPR 2017arXiv

citations

Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet

CVPR 2017arXiv

citations

A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects

CVPR 2018arXiv

citations

Attentive Fashion Grammar Network for Fashion Landmark Detection and Clothing Category Classification

CVPR 2018

citations

Human-Centric Indoor Scene Synthesis Using Stochastic Grammar

CVPR 2018arXiv

citations

Inferring Shared Attention in Social Scene Videos

CVPR 2018

citations

Where and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks

CVPR 2018

citations

Learning Descriptor Networks for 3D Shape Synthesis and Analysis

CVPR 2018arXiv

citations

Interpretable Convolutional Neural Networks

CVPR 2018arXiv

citations

Learning Generative ConvNets via Multi-Grid Modeling and Sampling

CVPR 2018arXiv

citations

RAVEN: A Dataset for Relational and Analogical Visual REasoNing

CVPR 2019

citations

Reasoning Visual Dialogs With Structural and Partial Observations

CVPR 2019

citations

Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model

CVPR 2019

citations

Unsupervised Disentangling of Appearance and Geometry by Deformable Generator Network

CVPR 2019

citations

Joint Training of Variational Auto-Encoder and Latent Energy-Based Model

CVPR 2020arXiv

citations

Inducing Hierarchical Compositional Model by Sparsifying Generator Network

CVPR 2020arXiv

citations

Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

CVPR 2021arXiv

citations

ACRE: Abstract Causal REasoning Beyond Covariation

CVPR 2021arXiv

citations

Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis

CVPR 2021arXiv

citations

Learning Triadic Belief Dynamics in Nonverbal Communication From Videos

CVPR 2021arXiv

citations

Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution

CVPR 2021arXiv

citations

Diffusion-Based Generation, Optimization, and Planning in 3D Scenes

CVPR 2023arXiv

citations

Mining And-Or Graphs for Graph Matching and Object Discovery

ICCV 2015

citations

Attributed Grammars for Joint Estimation of Human Attributes, Part and Pose

ICCV 2015

citations

Automated Facial Trait Judgment and Election Outcome Prediction: Social Dimensions of Face

ICCV 2015

citations

Predicting Human Activities Using Stochastic Grammar

ICCV 2017arXiv

citations

Jointly Recognizing Object Fluents and Tasks in Egocentric Videos

ICCV 2017

citations

Monocular 3D Human Pose Estimation by Predicting Depth on Joints

ICCV 2017

citations

Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning

ICCV 2019

citations

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

ICCV 2019

citations

Holistic++ Scene Understanding: Single-View 3D Holistic Scene Parsing and Human Pose Estimation With Human-Object Interaction and Physical Commonsense

ICCV 2019

citations

YouRefIt: Embodied Reference Understanding With Language and Gesture

ICCV 2021arXiv

citations

Spatio-Temporal Self-Supervised Representation Learning for 3D Point Clouds

ICCV 2021arXiv

citations

VLGrammar: Grounded Grammar Induction of Vision and Language

ICCV 2021arXiv

citations

X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events

ICCV 2023

citations

ARNOLD: A Benchmark for Language-Grounded Task Learning with Continuous States in Realistic 3D Scenes

ICCV 2023arXiv

citations

A Competence-aware Curriculum for Visual Concepts Learning via Question Answering

ECCV 2020

citations

Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference

ECCV 2020

citations

LEMMA: A Multi-view Dataset for LEarning Multi-agent Multi-task Activities

ECCV 2020

citations

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning

ECCV 2022

citations

Generative Hierarchical Learning of Sparse FRAME Models

CVPR 2017

citations

METASCENES: Towards Automated Replica Creation for Real-world 3D Scans

CVPR 2025

citations

Social World Model-Augmented Mechanism Design Policy Learning

NeurIPS 2025

citations

ProAgent: Building Proactive Cooperative Agents with Large Language Models

AAAI 2024

citations

An Embodied Generalist Agent in 3D World

ICML 2024

citations

Fast Peer Adaptation with Context-aware Exploration

ICML 2024

citations

Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning

ICML 2024

citations

Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation

NeurIPS 2018

citations

Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

NeurIPS 2019

citations

Learning Perceptual Inference by Contrasting

NeurIPS 2019

citations

PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points

NeurIPS 2019

citations

Learning Latent Space Energy-Based Prior Model

NeurIPS 2020

citations

Robust Visual Reasoning via Language Guided Neural Module Networks

NeurIPS 2021

citations

Unsupervised Foreground Extraction via Deep Region Competition

NeurIPS 2021

citations

On Path Integration of Grid Cells: Group Representation and Isotropic Scaling

NeurIPS 2021

citations

Iterative Teacher-Aware Learning

NeurIPS 2021arXiv

citations

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

NeurIPS 2022

citations

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

NeurIPS 2022

citations

EgoTaskQA: Understanding Human Tasks in Egocentric Videos

NeurIPS 2022

citations

Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

NeurIPS 2022

citations

Emergent Graphical Conventions in a Visual Communication Game

NeurIPS 2022

citations

MATE: Benchmarking Multi-Agent Reinforcement Learning in Distributed Target Coverage Control

NeurIPS 2022

citations

Learning non-Markovian Decision-Making from State-only Sequences

NeurIPS 2023

citations

Evaluating and Inducing Personality in Pre-trained Language Models

NeurIPS 2023

citations

Learning Energy-Based Prior Model with Diffusion-Amortized MCMC

NeurIPS 2023

citations

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models

NeurIPS 2023

citations

Diplomat: A Dialogue Dataset for Situated PragMATic Reasoning

NeurIPS 2023

citations

A Theory of Generative ConvNet

ICML 2016

citations

Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction

ICML 2018

citations

Song-Chun Zhu

Papers (81)

CLOVA: A Closed-LOop Visual Assistant with Tool Usage and Update

Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage

Decompositional Neural Scene Reconstruction with Generative Diffusion Prior

Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis

Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World

Neural-Symbolic Recursive Machine for Systematic Generalization

Differentiable Information Enhanced Model-Based Reinforcement Learning

Joint Action Recognition and Pose Estimation From Video

Recognizing Car Fluents From Video

Inferring Forces and Learning Human Utilities From Videos

Multi-View People Tracking via Hierarchical Trajectory Composition

Mining Object Parts From CNNs via Active Question-Answering

CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet

A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects

Attentive Fashion Grammar Network for Fashion Landmark Detection and Clothing Category Classification

Human-Centric Indoor Scene Synthesis Using Stochastic Grammar

Inferring Shared Attention in Social Scene Videos

Where and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks

Learning Descriptor Networks for 3D Shape Synthesis and Analysis

Interpretable Convolutional Neural Networks

Learning Generative ConvNets via Multi-Grid Modeling and Sampling

RAVEN: A Dataset for Relational and Analogical Visual REasoNing

Reasoning Visual Dialogs With Structural and Partial Observations

Divergence Triangle for Joint Training of Generator Model, Energy-Based Model, and Inferential Model

Unsupervised Disentangling of Appearance and Geometry by Deformable Generator Network

Joint Training of Variational Auto-Encoder and Latent Energy-Based Model

Inducing Hierarchical Compositional Model by Sparsifying Generator Network

Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

ACRE: Abstract Causal REasoning Beyond Covariation

Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis

Learning Triadic Belief Dynamics in Nonverbal Communication From Videos

Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution

Diffusion-Based Generation, Optimization, and Planning in 3D Scenes

Mining And-Or Graphs for Graph Matching and Object Discovery

Attributed Grammars for Joint Estimation of Human Attributes, Part and Pose

Automated Facial Trait Judgment and Election Outcome Prediction: Social Dimensions of Face

Predicting Human Activities Using Stochastic Grammar

Jointly Recognizing Object Fluents and Tasks in Egocentric Videos

Monocular 3D Human Pose Estimation by Predicting Depth on Joints

Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

Holistic++ Scene Understanding: Single-View 3D Holistic Scene Parsing and Human Pose Estimation With Human-Object Interaction and Physical Commonsense

YouRefIt: Embodied Reference Understanding With Language and Gesture

Spatio-Temporal Self-Supervised Representation Learning for 3D Point Clouds

VLGrammar: Grounded Grammar Induction of Vision and Language

X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events

ARNOLD: A Benchmark for Language-Grounded Task Learning with Continuous States in Realistic 3D Scenes

A Competence-aware Curriculum for Visual Concepts Learning via Question Answering

Learning Multi-layer Latent Variable Model via Variational Optimization of Short Run MCMC for Approximate Inference

LEMMA: A Multi-view Dataset for LEarning Multi-agent Multi-task Activities

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning

Generative Hierarchical Learning of Sparse FRAME Models

METASCENES: Towards Automated Replica Creation for Real-world 3D Scans

Social World Model-Augmented Mechanism Design Policy Learning

ProAgent: Building Proactive Cooperative Agents with Large Language Models

An Embodied Generalist Agent in 3D World

Fast Peer Adaptation with Context-aware Exploration

Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning

Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation

Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

Learning Perceptual Inference by Contrasting

PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points

Learning Latent Space Energy-Based Prior Model

Robust Visual Reasoning via Language Guided Neural Module Networks

Unsupervised Foreground Extraction via Deep Region Competition

On Path Integration of Grid Cells: Group Representation and Isotropic Scaling

Iterative Teacher-Aware Learning

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

EgoTaskQA: Understanding Human Tasks in Egocentric Videos

Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

Emergent Graphical Conventions in a Visual Communication Game

MATE: Benchmarking Multi-Agent Reinforcement Learning in Distributed Target Coverage Control

Learning non-Markovian Decision-Making from State-only Sequences

Evaluating and Inducing Personality in Pre-trained Language Models

Learning Energy-Based Prior Model with Diffusion-Amortized MCMC

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models