A way to test this point is linearizing the response of the autoencoders in the low-contrast regime and check that it is shift invariant. Using a Taylor expansion, the response for low-contrast images can be approximated by the Jacobian around the origin (the zero-contrast image,
\({\boldsymbol 0}\), which is just a flat gray patch):
\begin{eqnarray}
{\boldsymbol y} &\; = & S_\theta ({\boldsymbol x}) \nonumber \\
{\boldsymbol y}_{\textrm {low}} &\; = & S_\theta ({\boldsymbol 0} + {\boldsymbol x}_{\textrm {low}})\nonumber\\
&\;\approx& S_\theta ({\boldsymbol 0}) + \nabla _{\!{\boldsymbol x}} S_\theta ({\boldsymbol 0}) \cdot {\boldsymbol x}_{\textrm {low}} \quad\\
{\boldsymbol y}_{\textrm {low}} &\; \approx & \nabla _{\!{\boldsymbol x}} S_\theta ({\boldsymbol 0}) \cdot {\boldsymbol x}_{\textrm {low}} \nonumber
\end{eqnarray}
where we assumed that the response for zero-contrast images is zero. If the behavior of the system at this low-energy regime is shift invariant, the Jacobian matrix can be diagonalized as
\(\nabla _{\!{\boldsymbol x}} S_\theta ({\boldsymbol 0}) = B\cdot \lambda \cdot B^{-1}\), with extended oscillatory basis functions in the columns of
\(B\) (and rows of
\(B^{-1}\)). Fourier basis and cosine basis are examples of extended (nonlocal) oscillatory functions that diagonalize shift invariant systems. The reason for this result is equivalent to the emergence of cosine basis when computing the principal components of stationary signals (shift-invariant autocorrelation) (
Clarke, 1981). As a result, the slope of the response for low-contrast sinusoids (the CSF) will be related to the eigendecomposition of the Jacobian of the system at
\({\boldsymbol 0}\). Let’s compute the response for a sinusoid in this Taylor/Fourier setting to see the relation. A basis function
\({\boldsymbol b}^f\) with specific frequency
\(f\) is orthogonal to all rows (sinusoids) in
\(B^{-1}\) except that of the same frequency, that is,
\(B^{-1} \cdot {\boldsymbol b}^f = {\boldsymbol \delta }^{f^{\prime }f}\). And this delta selects the corresponding column (of frequency
\(f\)) among all the columns in the matrix B:
\begin{eqnarray}
{\boldsymbol y}^f &\; = & S_\theta ({\boldsymbol b}^f) \approx \nabla _{\!{\boldsymbol x}} S_\theta ({\boldsymbol 0}) \cdot {\boldsymbol b}^f \nonumber \\
&\; \approx & B\cdot \lambda \cdot B^{-1} \cdot {\boldsymbol b}^f \nonumber \\
&\; \approx & B\cdot \lambda \cdot {\boldsymbol \delta }^{f^{\prime }f} \nonumber \\
&\; \approx & \lambda _f \, {\boldsymbol b}^f
\end{eqnarray}
So the slope of the response for basis functions of frequency
\(f\) is
\(\lambda _f\) (the corresponding eigenvalue of the Jacobian of the autoencoder). As a result, for systems with shift invariance in the low-contrast regime, the eigenvalues of the linear approximation of the system (eigenvalues of the Jacobian) are conceptually similar to the CSF. A direct comparison of the eigenvalue spectrum with the CSF may not be simple because the eigenfunctions may differ from Fourier sinusoids. Examples of this include isotropic systems (with a constant sensitivity for certain
\(|f|\) independent of orientation). In this case, the eigenbasis may be not sinusoids, but arbitrary linear combinations of sinusoids of the same frequency and different orientation.