I made a free software for all operative systems, even Android. It's called Deeplearning2C. It can train a neural network and generate C code and MATLAB-code. C-code for embedded systems and MATLAB-code for simulation.
I tried a example when I trained a neural network with linear data with constraints = nonlinear model in other words. It works very well. Here is the result when I use identity as activation functions.
I identification the model from the features (inputs)
$$-y(t-1), -y(t-2), ... , -y(t-n), u(t), u(t-1), ... , u(t-m)$$
And the label (output) $y(t)$. This is the regular transfer function or ordinary differential equation identification method.
But the issue I have is that I have a lack of experience of system identification when it comes to neural networks. I'm used to state space models (Subspace identification) and regular parameter estimation (Recursive Least Square). I wrote a library for that too.
My questions for you are that:
- What type of activation functions should I use?
- How many layers is "good" enough?
- How many neurons should I have?
- How do I know...how to know what to select? I'm seeking practical experience for identification with neural networks.
Here is my software Deeplearning2C. It's using Deeplearning4J as the core, but the user interface is made by me.
Also here is the MATLAB/GNU Octave library for linear system identification with recursive least square and subspace identification. I wrote a C-library too for recursive least square identification as well.