arXiv：cs.LG· El Mahdi Chayti, Martin Jaggi·· 13 小时前
Split-Client：异步曲率下的二阶优化
Second-Order Optimization with Asynchronous Curvature
AI 导读
Split-Client 采用任务并行设计，曲率客户端在后台分解 Hessian，梯度客户端用最新发布的矩阵执行三次正则化步，使曲率开销表现为延迟而非阻塞。针对延迟的精确 Hessian，研究证明了一阶平稳性界依赖于曲率的平均年龄而非最大年龄，并可在固定成本模型下达到与最优调参 Lazy Hessian 相同的墙钟时间扩展。
正文
Abstract:Second-order methods make fast progress per iteration, but forming and factorizing the Hessian often dominates their wall-clock time. We study \emph{Split-Client}, a task-parallel design in which a curvature client factorizes Hessians in the background while a gradient client takes cubic-regularized steps with the most recently published matrix, so that curvature cost appears as a delay rather than as blocking time. For delayed exact Hessians, we prove a first-order stationarity bound that depends on the average age of the curvature rather than its maximum, using an overlap count and a weighted Hardy inequality that allows any bounded-error initial matrix; the analysis extends to subsampled and uniformly inexact curvature. Under a fixed-cost model, the resulting wall-clock bound matches the scaling of optimally tuned Lazy Hessian without choosing a reuse period. In experiments on two LIBSVM datasets, an implementation decreases the objective earlier in wall-clock time than Lazy and Vanilla cubic Newton, with the largest gains at intermediate accuracy.
| Comments: | Old title: A Split-Client Approach to Second-Order Optimization |
| Subjects: | Optimization and Control (math.OC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.15714 [math.OC] |
| (or arXiv:2510.15714v4 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2510.15714 arXiv-issued DOI via DataCite |
Submission history
From: El Mahdi Chayti [view email]
[v1]
Fri, 17 Oct 2025 14:58:11 UTC (99 KB)
[v2]
Wed, 17 Dec 2025 19:22:11 UTC (158 KB)
[v3]
Thu, 14 May 2026 19:46:07 UTC (155 KB)
[v4]
Fri, 9 Oct 2026 17:05:28 UTC (176 KB)
来源：arXiv：cs.LG · arxiv.org