PEIRA · Method

Align views through an optimal ridge regressor

Predict one view from the other with a linear regressor, solved in closed form, and maximize its trace.

The PEIRA objective Two views x and y of one image go through encoders U and V. The features define two statistics: Sigma, the covariance across views (shared signal), and N, the covariance inside each view (within-view variability). The optimal ridge regressor between views is P = Sigma (N + lambda I) inverse, a signal-to-noise ratio between views. PEIRA maximizes the trace of P, with a norm penalty on U and V: it minimizes minus one half Tr(P) plus lambda over two times the sum of the squared norms of U and V. Simplified diagram, for fixed encoders x U U(x) two views of one image y V V(y) features in ℝᵏ Σ_{U,V} Shared signal covariance across views N_{U,V} Within-view variability covariance inside each view Optimal ridge regressor P_{U,V} = Σ_{U,V}(N_{U,V} + λI)^{−1} signal-to-noise ratio between views PEIRA maximizes the trace of P with a norm penalty on U, V ℰ_{PEIRA}(U,V) = −½ Tr(P_{U,V}) + ½λ(‖U‖_{2}^{2} + ‖V‖_{2}^{2}) Tr(P) sums the contributions of all eigen-modes of the regressor. The paper also gives a stochastic training algorithm with EMA statistics and an auxiliary loss.

Two views x and y of one image go through encoders U and V.

0:00 / 0:16