Since this appears to be homework, I can give you a few generic pointers
LTI systems maintain periodicity: if the input is periodic so is the output and vice versa.
If a signal is finite in length it can't be periodic
If a signal has finite bandwidth, it must be infinite in time.
You can model the input as $x(t)$ as cosine multiplied with a rectangular ...
This is actually a math problem, but it is related to signal processing via the concept of signal space. Since it's a homework-style problem you'll only get hints that should help you solve the problem on your own.
verify that the given vectors are orthonormal.
if you've convinced yourself that the vectors are orthonormal, you can conclude that the ...