Pierre Ablin

7

Papers

61

Total Citations

Papers (7)

Theory, Analysis, and Best Practices for Sigmoid Self-Attention

The AdEMAMix Optimizer: Better, Faster, Older

Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency

Optimization without Retraction on the Random Generalized Stiefel Manifold

Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging

Careful with that Scalpel: Improving Gradient Surgery with an EMA

How Smooth Is Attention?