I know the basic working principle of sinusoidal oscillators. But I have a doubt on this matter when I came across Gram-Schmidt orthogonalization procedure. I have read that the source for sinusoid signal output from an oscillator is the noise. Due to controlled positive feedback a single frequency component is selected and amplified, right? If the noise is a white noise, we say it has all frequency components. We think in terms of all possible sinusoids with different frequencies. When coming to frequency response too, we think in terms of sinusoids. Why can't we think in terms of other orthogonal basis set? Although it makes life difficult, I would like to know the justification for thinking in terms of sinusoids. Is sinusoids the nature's default signals?
Time invariance plays a huge role in nature. Most systems (including your ear/brain) don't have an absolute time reference but treat all points in time equally. That results in a preference for the description of these systems with essentially time invariant basis functions, which is what (complex) sinusoids are.
For linear time invariant systems, the complex exponentials are even eigenfunctions of the system and allow you to describe the system as a "diagonal" operator, i.e. in a very simple way. That even includes a system that just performs a time delay.
Nonlinear time invariant systems can also be described in terms of sinusoids, but it gets way more complicated there. A nonlinear time invariant system excited by a periodic input will produce a periodic output in general, and you can analyze both in terms of sinusoids.
But you can also perform a more mathematical rigorous analysis by looking at the nonlinear system response to small disturbances of any input signal. And taking sinusoids to describe these disturbances gives you a local description in a time invariant way.
So all in all, if you have a system that is time invariant, no matter of what nature, time shift invariant basis functions are helpful, and complex sinusoids are precisely that.
Complex exponentials (with decaying sinusoids being the real part) are the solutions to certain types of low-order linear differential equations. Modeling simple natural phenomena with these low-order linear differential equations turns out to be surprisingly useful. "Why did the real world turn out this way?" might be a good question for philosophers.
Mandlebrot and Talib might claim that believing that these simplified models actually accurately apply to real-world systems bigger than can fit in a freshman physics lab is just a trick of the human mind.
Sinusoids is just a different way of looking at what we have here (since frequency analysis is equivalent to time analysis) - it only is more useful in some cases. It's a model though, and that's why we often add noise models - because they help describe what sinusoids don't describe in an efficient way. And since we use the noise models so often, it's not that perfect.
You might want to look at mathematical relationships between frequency and time analysis to see why we often consider sinusoids.