Percy Liang

53

Papers

1,451

Total Citations

Papers (53)

Certified Defenses for Data Poisoning Attacks

NeurIPS 2017arXiv

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Large Language Models as Analogical Reasoners

On the Learnability of Watermarks for Language Models

BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments

Benchmarking and Improving Generator-Validator Consistency of Language Models

Unsupervised Risk Estimation Using Only Conditional Independence Structure

NeurIPS 2016arXiv

Learning Overcomplete HMMs

NeurIPS 2017arXiv

Model Equality Testing: Which Model is this API Serving?

Establishing Best Practices in Building Rigorous Agentic Benchmarks

Unsupervised Transformation Learning via Convex Relaxations

NeurIPS 2017arXiv

Reliable and Efficient Amortized Model-based Evaluation

Position: In-House Evaluation Is Not Enough. Towards Robust Third-Party Evaluation and Flaw Disclosure for General-Purpose AI

Calibrated Structured Prediction

On-the-Job Learning with Bayesian Decision Theory

MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation

Position: A Safe Harbor for AI Evaluation and Red Teaming

Position: On the Societal Impact of Open Foundation Models

Learning with Relaxed Supervision

Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models

Estimating Mixture Models via Mixtures of Polynomials

Cheaply Estimating Inference Efficiency Metrics for Autoregressive Transformer Models

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining

Holistic Evaluation of Text-to-Image Models

Reified Context Models

Learning Fast-Mixing Models for Structured Prediction

Estimation from Indirect Supervision with Linear Moments

Understanding Black-box Predictions via Influence Functions

Developing Bug-Free Machine Learning Systems With Formal Mathematics

World of Bits: An Open-Domain Platform for Web-Based Agents

Convexified Convolutional Neural Networks

Fairness Without Demographics in Repeated Loss Minimization

On the Relationship between Data Efficiency and Error for Uncertainty Sampling

Semidefinite relaxations for certifying robustness to adversarial examples

Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss

A Retrieve-and-Edit Framework for Predicting Structured Outputs

Unlabeled Data Improves Adversarial Robustness

SPoC: Search-based Pseudocode to Code

On the Accuracy of Influence Functions for Measuring Group Effects

Verified Uncertainty Calibration

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?

Diffusion-LM Improves Controllable Text Generation

Improving Self-Supervised Learning by Characterizing Idealized Representations

Insights into Pre-training via Simpler Synthetic Tasks

Decentralized Training of Foundation Models in Heterogeneous Environments

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Deep Bidirectional Language-Knowledge Graph Pretraining

PRODIGY: Enabling In-context Learning Over Graphs

Lexinvariant Language Models

AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

Data Selection for Language Models via Importance Resampling

Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes