How to calculate signal which is not changed by a filter?

Question

Suppose that there is a FIR filter F and a signal S. The filtered signal is the convolution of F and S, F * S.

The problem: how to calculate a signal S' such that F * S' = S' (the filtered version is the same as itself, up to a fixed time shift), and S' minimizes norm(S, S') (where norm is some kind of distance metric, for example sum of squared differences).

Loosely: how to calculate the most similar signal to a given signal which is not affected by a filter?

EDIT: The original problem may be over constrained because there may be very few (or no) signals for which F*S = S exactly. An alternate formulation is to find S' which minimizes the combined error alpha*norm( F*S' - S') + beta*norm( S' - S ), for some weights alpha and beta.

EDIT: The specific filter I have in this case is F=[ 0.00097656, -0.00976562, 0.06347656, -0.15625 , 0.18554688, 0.83203125, 0.18554688, -0.15625 , 0.06347656, -0.00976562, 0.00097656] which is a fairly modest lowpass filter. I was interested in the general case as well though.

Answer 1

If a filter with transfer function $H(f)$ passes a signal $x(t)$ with Fourier transform $X(f)$ unchanged, then $H(f)X(f) = X(f)$ for all $f$, and so $H(f)$ must have the property that $$H(f) = 1 ~ \text{for all frequencies}~ f~ \text{for which}~ X(f) \neq 0.$$ Thus, an arbitrary filter will not enjoy this property for very many signals, perhaps for no signal at all. Indeed, if $H(f_0) = H(-f_0) = 1$ for some $f_0$, then the filter will pass the sinusoid $A\cos(2\pi f_0t + \theta), -\infty < t < \infty$ unchanged, but it will not pass a sinusoidal pulse such as $$x(t) = \begin{cases} A\cos(2\pi f_0t + \theta), & -T < t < T,\\ 0, &\text{otherwise,}\end{cases}$$ without distorting it. Of course, it is possible that $H(f) \neq 1$ for all $f$ and so even passing a monochromatic sinusoid is not possible (cf. @hotpaw's comment). In short, the subspace referred to by @AtulIngle might well be the trivial subspace of zero dimension containing only the zero signal. It is also worth noting that if $H(f)$ has constant nonzero value over an interval, then the filter is physically unrealizable for essentially the same reason that an ideal low-pass filter is unrealizable.

So, the OP's question is likely to have no answer at all because the signals $S^\prime$ that he seeks to find may not exist at all for the given filter. Perhaps something else was intended to be asked?