0
$\begingroup$

I read the paper Deep Feature Flow for Video Recognition https://arxiv.org/abs/1611.07715.

In Sec.3, the author implements bilinear interpolation like this:

$$f_i^c(p)=\sum\limits_{q}G(q,p+\delta p)f_k^c(q) \tag{1}$$

Where $q$ is the point from the source image, and $p$ is the points on the target image. $\delta p$ is the distance the point moved each point $p$ (not $\delta \bullet p$). $G$ is defined as

$$G(q,p+\delta p)=g(q_x,p_x+\delta p_x)g(q_y,p_y+\delta p_y)\tag{2}$$

And the bilinear interpolation is defined in wiki as:

$$f(x,y)\approx {\frac {y_{2}-y}{y_{2}-y_{1}}}f(x,y_{1})+{\frac {y-y_{1}}{y_{2}-y_{1}}}f(x,y_{2})\tag{3}$$

I think the operation $(1)$ and $(3)$ is equivalent. How can I derive the filter $(1)$ from $(3)$?

$\endgroup$
  • 1
    $\begingroup$ Can you please clarify if you are asking for a verification of the steps taken or how to actually do it? (Or what are the implications to the subject of the paper? ). What is the question in this case? $\endgroup$ – A_A Nov 10 '18 at 11:08
  • $\begingroup$ @A_A Sorry for confusing you. What I mean is, it seems that operation $(1)$ (applying the filter $G$ on the image $f$) and $(3)$ is equivalent, how can I derive the filter (1) from (3)? $\endgroup$ – Huang Yuheng Nov 10 '18 at 14:07
1
$\begingroup$

Honestly I haven't read that article you linked to, but as long as you want a convolution kernel for 2D bilinear interpolation, then the following should help.

Bilinear interpolation gives a crude result which can be sufficient in case the application does not require a perfect output otherwise. Then the following simple triangular 2D convolution kernel implements a bilinear interpolation of the ratio (after compression by $M$) given by $r = \frac{L}{M} > 1$.

L = 9;          % upsampling ratio
M = 5;          % downsampling ratio (M < L assumed)

h   = (1/L)*conv(ones(1,L),ones(1,L));  % 1D linear interpolator.
hBL = h'*h;                             % 2D bi-linear interpolator.

I = im2double(imread('Cameraman.tif')); % Read some image into double
S = size(I);                            % size of image

Ie = zeros( L*S(1), L*S(2) ) ;          % expand the image by stuffing zeros
Ie( 1:L:end , 1:L:end ) = I;            % assign the original pixels

Iup = conv2(Ie,hBL);                    % linear upsample (interpolate) by L
Iint = Iup(1:M:end,1:M:end);            % downsample by M

figure,imshow(I);title('original image');
figure,imshow(Iint);title('upsampled by L/M times');
$\endgroup$
  • $\begingroup$ Thank you for your code. I want to know why 1D linear interpolator kernel is h = (1/L)*conv(ones(1,L),ones(1,L)); and why 2D bi-linear interpolator kernel is h'*h? Can you give some mathematical derivation? $\endgroup$ – Huang Yuheng Nov 13 '18 at 2:55
  • $\begingroup$ @HuangYuheng Hi! To see how to derive a 1D linear interpolation kernel, you can read Discrete-Time Signal Processing 2e, A.Oppenheim et al, ch-4, sec-4.6, p-175, Eq-4.92 . To see how to get 2D impulse response from 1D, you can read 2D signal and Image processing J.S.Lim, ch-1&4, Finite Impulse Response Filter Design (look specifically for separable filters) $\endgroup$ – Fat32 Nov 13 '18 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.