Given the real-valued functions $f_1$ and $f_2$ with $x\in\mathbb{R}$, then $$ \frac{df_2(f_1(x))}{dx} = \frac{df_2(f_1(x))}{d f_1(x)}\frac{df_1(x)}{dx}$$

Is it then the case that if we also have a real-valued function $f_3$, then $$ \frac{df_3(f_2(f_1(x)))}{dx} = \frac{df_3(f_2(f_1(x)))}{df_2(f_1(x))}\frac{df_2(f_1(x))}{df_1(x)}\frac{df_1(x)}{dx}$$

Is this correct?

If so, does the chain rule generalize to the following statement: $$ \frac{df_{N}(f_{N-1}(...f_2(f_1(x))))}{dx} $$ $$= \frac{df_{N}(f_{N-1}(...f_2(f_1(x))))}{df_{N-1}(...f_2(f_1(x)))} \frac{df_{N-1}(...f_2(f_1(x)))}{df_{N-2}(...f_2(f_1(x)))}...\frac{df_2(f_1(x))}{df_1(x)}\frac{df_1(x)}{dx}$$

My apologies about this looking like a straight calculus question. I am asking to figure out the backpropagation algorithm in its general form.

  • $\begingroup$ that's true.... $\endgroup$ – Fat32 Jun 18 '17 at 1:59
  • $\begingroup$ it's fine with me to post it here, but there is a nice mathematics SE which would be an ideal place for this post. $\endgroup$ – robert bristow-johnson Jun 18 '17 at 3:58

I'm not a mathematician but if the multiple composition $ f_1(f_2(f_3 \dots)))$ is equivalent to a sequence of single compositions $g_2(g_1)$, and another $g_3(g_2)$ and so forth.

The generalization looks reasonable for $x, f_{1}, f_{2} \dots f_N \in \cal{R}$ and $f_i$ differentiable with no discontinuities.

| improve this answer | |
  • $\begingroup$ I'm far from a mathematician myself, and I think the three responses saying it is correct is good enough for me. :) $\endgroup$ – Josh Jun 18 '17 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.