Suppose that I need to apply a convolution filter to an image. I want to do it by using the convolution theorem so I compute the kernel for the size of the input image to later calculate fft's and multiply. But I want the fft calculation to be less time consuming. The kernel loses its magnitude the further from its center, as usual. If I want to crop it, say, have a smaller kernel with only significant values, can I for instance compute its fft and later somehow extend the resulting frequency domain image of the kernel to fit the size of the input image, to later multiply them point-wise?
If you want to do the convolution in the frequency domain, you have to pad the kernel with zeros to make it the same size as the image. There is really no way around it. The upside is that you only need to take the FFT of the filter once, and then you can re-use it for multiple images.
If you are concerned with speed, you would have to do some analysis here. For small separable kernels doing the convolution in the spatial domain may be faster.
It depends on what kind of kernel you are applying. In the paper of SURF, the authors use an approximation to the second order Gaussian partial derivative kernel, see the 3rd and 4th figure below.
By such approximation, you can expand the kernel (or change the scaling) by just adding elements with single numbers. See the figure shown below.