Perform Transposed Convolution in Spectral / Frequency Domain?

Question

I'm doing some experimentation on performing end to end generative modeling in the frequency domain. I've got a working convolutional layer, but do not yet have a Conv2DTranspose equivalent. Please note this is not deconvolution! Unfortunately, my signal processing knowledge is pretty weak at the moment (I'm taking some classes in SP this Fall).

I know convolution is performed in the frequency domain by performing pointwise multiplication. Does transpose convolution have a frequency domain equivalent? If so, what is the correct operation or series of operations to perform here?

Answer 1

Basically, if we define convolution as $ y = h \ast x $, it can be written in Matrix form (See Generate the Matrix Form of 1D Convolution Kernel):

$$ \boldsymbol{y} = H \boldsymbol{x} $$

Transposed Convolution is given by:

$$ {H}^{T} \boldsymbol{z} $$

If you look carefully, you'd see the spatial operation is basically correlation instead of convolution (Namely the kernel isn't flipped).

To achieve that in Frequency Domain you need to multiply by the conjugate of the kernel in Frequency domain instead of the kernel itself.

The tricky part is the dimensions. It will work as I described in Replicate MATLAB's conv2() in Frequency Domain.

Pay attention that in the context of Deep Learning the whole idea of the operation is that the kernel will be learned (Adaptively in each back propagation iteration). This is in order to learn the best kernel for up sampling operation.

References

• What is the difference between UpSampling2D and Conv2DTranspose functions in keras?
• An Introduction to Different Types of Convolutions in Deep Learning.