时光荏苒

推荐论文：

An Exploration of Non-Euclidean Gradient Descent_ Muon and its Many Variants

Back to Basics_ Let Denoising Generative Models Denoise

Branching Flows_ Discrete, Continuous, and Manifold Flow Matching with Splits and Deletions

Decoupling Positional and Symbolic Attention Behavior in Transformers

How Memory in Optimization Algorithms Implicitly Modifies the Loss

Isotropic Curvature Model for Understanding Deep Learning Optimization_ Is Gradient Orthogonalization Optimal？

L2M_ Mutual Information Scaling Law for Long-Context Language Modeling

Larger Datasets Can Be Repeated More_ A Theoretical Analysis of Multi-Epoch Scaling in Linear Regression

On the Structure of Floating-Point Noise in Batch-Invariant GPU Matrix Multiplication

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling

Scaling Laws and In-Context Learning_ A Unified Theoretical Framework

https://papers.cool/arxiv/2510.09827,2511.13720,2511.09465,2511.11579,2502.02132,2511.00674,2503.04725,2511.13421,2511.00025,2505.22491,2511.06232

妈妈“现在的孩子怎么这么多病呢”
儿子“不是病多了，是能治的病多了，搁以前都是夭折的”

好精辟的回答，受教了！

来源：https://www.zhihu.com/question/1926923396882621109/answer/1970943451643224638 评论区

入职两年多，第二次到北京总部。

推荐论文：

Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization

AlphaFlow_ Understanding and Improving MeanFlow Models

Arithmetic-Mean μP for Modern Architectures_ A Unified Learning-Rate Scale for CNNs and ResNets

Equilibrium Matching_ Generative Modeling with Implicit Energy-Based Models

From Condensation to Rank Collapse_ A Two-Stage Analysis of Transformer Training Dynamics

On residual network depth

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization

Optimal Scaling Needs Optimal Norm

Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks

Who Said Neural Networks Aren't Linear？

Why Low-Precision Transformer Training Fails_ An Analysis on Flash Attention

https://papers.cool/arxiv/2510.04988,2510.20771,2510.04327,2510.02300,2510.06954,2510.03470,2510.19953,2510.03871,2510.11354,2510.08570,2510.04212

疯狗逻辑：虽然某些人跟疯狗有很大区别，但我只要我认为这些区别不重要，那么某些人就是疯狗。

我是一个比较蠢的人，只会按部就班地进行推导，同时也没啥直觉，通常无法理解能推导出来以外的内容。

推荐论文：

Conda_ Column-Normalized Adam for Training Large Language Models Faster

DiVeQ_ Differentiable Vector Quantization Using the Reparameterization Trick

Efficient Hyperparameter Tuning via Trajectory Invariance Principle

Muon Outperforms Adam in Tail-End Associative Memory Learning

Power Lines_ Scaling Laws for Weight Decay and Batch Size in LLM Pre-training

Unveiling the Role of Learning Rate Schedules via Functional Scaling Laws

https://papers.cool/arxiv/2509.24218,2509.26469,2509.25049,2509.26030,2505.13738,2509.19189

推荐论文：

Are We Really Learning the Score Function？ Reinterpreting Diffusion Models Through Wasserstein Gradient Flow Matching

Attention as an Adaptive Filter

Causal Attention with Lookahead Keys

Depth-Aware Initialization for Stable and Efficient Neural Network Training

Dynamic Low-rank Approximation of Full-Matrix Preconditioner for Training Generalized Linear Models

Flow Straight and Fast in Hilbert Space_ Functional Rectified Flow

Limitations of Normalization in Attention Mechanism

Predicting the Order of Upcoming Tokens Improves Language Modeling

Rotational Equilibrium_ How Weight Decay Balances Learning Across Neural Networks

Scaled-Dot-Product Attention as One-Sided Entropic Optimal Transport

The Optimiser Hidden in Plain Sight_ Training with the Loss Landscape's Induced Metric

Transition Models_ Rethinking the Generative Learning Objective

UltraMemV2_ Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning

Understanding Transformers through the Lens of Pavlovian Conditioning

https://papers.cool/arxiv/2509.00336,2509.04154,2509.07301,2509.05018,2508.21106,2509.10384,2508.17821,2508.19228,2305.17212,2508.08369,2509.03594,2509.04394,2508.18756,2508.08289

推荐论文：

Accelerating Newton-Schulz Iteration for Orthogonalization via Chebyshev-type Polynomials

Zero-Variance Gradients for Variational Autoencoders

https://papers.cool/arxiv/2506.10935,2508.03587

mark：知乎关注8万。