You can have all the taps be zero except for the last tap and the filter will delay the signal by that number of taps (as an all-pass). This doesn't do any filtering, and it is the actual filtering requirement that would drive the number of taps needed in the filter. For that requirement alone a dual port memory is a good solution for very long delays.
The delay for any filter is proportional to the transition bandwidth of that filter, sharper transitions in frequency require more memory (think moving averages and how the noise of a 10 day moving average of stock prices compares to a 90 day moving average). The sharpest possible transition in Hz achievable for a FIR filter is done with the moving average filter and would be the inverse of the time duration of the filter in seconds, as illustrated by the frequency response of a rectangular window (which that FIR filter would be). So for the time duration given in samples as $N$, and the transition bandwidth normalized the sampling rate $f_s$, the transition would be $1/N$ cycles/sample, or in radian frequency as in the plot below $2\pi/N$ radians/sample. For a bandpass equivalent of this we simply heterodyne the coefficients using $e^{j\omega n}$ to be rotating at the center frequency of interest. This filter, as a Sinc response suffers from signficant sidelobes, so is further improved by windowing which reduces the sidelobes but increases the transition width further (by a typical factor between 2 and 5 depending on window chosen). Any other FIR filter design techniques (least squares etc) would have the same result--so the rectangular case tells us the best that can be achieved (with an FIR filter). For much tighter transitions with less coefficients-- which then means much longer delay for less coefficients, an IIR filter can be used.
Bottom line for the delay associated with any filtering:
Tight transition bandwidth = Long Delay. The least delay that can be achieved with an FIR for a given transition bandwidth is the case of a simple moving average, where the number of taps $N$ required is the inverse of the transition bandwidth, and the associated delay is $N/2$. Any practical FIR filters are likely to have a factor 2 to 5 times larger than this.
Given this:
I want to distortionless-free delay a bandwidth of fs/100 and don't
care about the filter response outside of that band
The $fs/100$ bandwidth requirement specifies that the FIR filter needs to be at least 100 taps, longer by a factor of 2-5 depending on actual passband and stopband requirements. Beyond that zeros can be used to shift the time with no other consequence.