How to Estimate a Multi Channels and Multi Kernels Convolution Kernel (Deep Learning Style) Given the Input and Output Images

Question

Is it possible that can estimate convolutional kernel that have multi channels and multi filters ?

I saw answer from this to link to estimate kernel for one channel and one filter (Estimating Convolution Kernel from Input and Output Images). When I tried to implement it, I found that it cannot do correlation because the channel of output is not equal with the channel of input.

For example of data,

Input = 20 x 20 x 3 output = 20 x 20 x 32 kernel size = 3 x 3 x 3 x 32

Answer 1

It is possible as things are staying linear and spatially invariant.

First, you can basically solve 32 times the same problem of input of size 20x20x3 filtered by 3x3x3 filter with output of size 20x20.

There are several computational approaches to solve this:

• Build the sparse convolution matrix of the kernel and follow Estimating Convolution Kernel from Input and Output Images.
• Use the operators form and solve this using Conjugate Gradient as in Automatic Image Enhancement of Images of Scanned Documents (Auto Whitening).
• Use Deep Learning framework and use the Automatic Differentiation engine to utilize Gradient Descent based solution.