The Matrix Form of a 2D Circular Convolution

Question

I have 3 closely related questions regarding 2d convolutions and how they are represented in matrix form.

1. Miming what happens in 1d, I assume the product of a doubly block circulant matrix $A$ by a (vectorized) image $x$ by can be understood as a circular convolution with a kernel $a$. That is $$ Ax= a \star x. $$ Is this right?

2. When differentiating the cost function $\frac12\|Ax-y\|_2^2$ we obtain $A^T(Ax-y)$. If $A$ is doubly block circulant, how can this last expression be mapped to the circular convolution language? What does $A^TAx$ and $A^Ty$ mean in terms of circular convolutions?

3. What's a good reference (book or paper) on this subject? meaning that I want to interpret 1d/2d convolutions and Fourier transforms using linear algebra.

Answer 1

1. Yes, indeed. You may represent the convolution in a Matrix Form. Pay attention that this form assumes the image is column / row stacked into a vector.

2. If you're after a circular convolution, you may use DFT matrix to diagonalize the matrix and then simplify the equations. Have a look at Circular Convolution Matrix of $ {H}^{H} {H} $.

3. I am not aware of books on the subject. But I have written many answers on it in this site:

   • Circular Convolution Matrix of $ {H}^{H} {H} $.
   • Applying a 2D Convolution Using 2D FFT.
   • Generate the Matrix Form of 2D Convolution Kernel.
   • Generate the Convolution Matrix of 2D Kernel for Convolution Shape of same.