When a time-shift is applied to a signal, it only changes the angles in the DFT coefficients.
In turn, the time-shift used can determined by observing how the spectral line angles have been changed according to their index $k$.
Experiment
(I'm using the general term DFT, the transform implemented by FFT algorithm.)
The following plots show a signal, its DFT magnitude and angle, and the same for a shifted version.


This answers your first question: Time-shifting a signal results in a change of phase in the spectral lines.
For the second question, how to determine the time-shift from the DFT, let's look at how the time-shift changes the DFT.
A phase shift is a multiplication by a complex exponential
As you mentioned, a change of phase is a multiplication by a complex exponential. This is consistent with the the relationship:
$$\cos(\omega t + \varphi) = \Re \{ e^{j (\omega t + \varphi) } \} = \Re \{ e^{j \omega t}e^{j\varphi} \}$$
where adding $\varphi$ and multiplying by $e^{j\varphi}$ are equivalent.
Time-shift DFT pair
This pair of formulas relates the effect of time-shifting in the time-domain and in the frequency-domain:
$$y[n]=x[((n-n_{d}))_{N}]\overset{\textrm{DFT}}{\longleftrightarrow}Y[k]=e^{-j\frac{2\pi k}{N}n_{d}}X[k]$$
for a signal of length $N$ and a shift of $n_d$ samples.
Angle increase is different for each spectral line
To get the DFT of a shifted signal the DFT of the unshifted signal $X[k]$ is multiplied by $e^{-j\frac{2\pi k}{N}n_{d}}$.
Let's note $\frac{2\pi k}{N}n_{d}$ is a value proportional both to:
- The ratio $n_d/N$. This determines a constant angle value.
- The index of the spectral line $k$. Since the phase angle corresponding to a given time is proportional to the frequency, this angle increases for each spectral line.
This multiplication by a variable factor applies the same amount of time-shift to all spectral lines, and thus the whole signal is consistently time-shifted.
Now if the angles for the original signal are subtracted from the angles for the shifted signal, the result is an arithmetical progression with a common difference $p_d$:

Getting the time-shift $n_d$ from the common difference:
$$n_d = -\frac {N} {2 \pi}p_d$$
Application
This piece of Python code plots the images above and computes the time delay:
import numpy as np
import scipy.fft as sf
import matplotlib.pyplot as plt
# Generate a signal delayed or not
def signal_gen(length, delay=0, aliasing=False):
indices = np.arange(-delay, -delay+length)
if aliasing: indices = indices % length
values = np.linspace(1, 0, length) ** 2 + 0.2
xn = np.where((indices<0) | (indices>=length), 0, values[indices])
return xn
# Plot signal
def plot(xn, Xk, title='Signal', unwrap=False):
# Plot signal
fig, axes = plt.subplots(ncols=3, squeeze=True, figsize=(8, 2.5), layout='constrained')
ax = axes[0]
ax.set_title(title)
ax.stem(xn)
# Plot DFT magnitude
ax = axes[1]
ax.set_title('DFT magnitude')
ax.stem(abs(Xk))
# Plot DFT angle
ax = axes[2]
ax.set_title('DFT angle')
angles = np.angle(Xk)
if unwrap: angles = np.unwrap(angles)
ax.stem(angles)
N = 9 # number of samples to create
nd = 4 # time-delay in samples
# Original and time-shifted signals
xn = signal_gen(N, 0)
xn_s = signal_gen(N, nd, aliasing=True)
# Plot
Xk, Xk_s = [sf.fft(signal) for signal in [xn, xn_s]]
plot(xn, Xk, title='Unshifted signal')
plot(xn_s, Xk_s, title='Time-shifted signal')
# Phase differences between DFT of shifted and unshifted signals
phase_wrapped = np.angle(Xk_s) - np.angle(Xk)
phase = np.unwrap(phase_wrapped)
# Print wrapped and unwrapped phase differences and increments
print(np.array2string(phase_wrapped, formatter={'float_kind': lambda x: "%.2f" % x}))
print(np.array2string(phase, formatter={'float_kind': lambda x: "%.2f" % x}))
print(np.array2string(np.diff(phase), formatter={'float_kind': lambda x: "%.2f" % x}))
# Compute time-delay
pd = phase[1]-phase[0]
nd2 = -N * pd / (2*np.pi)
print(f'nd = {nd2:.2f}')
It outputs
The time-delay as a number of samples is determined from the phase increment $p_d=-2.79$:
nd = 4.00
You may want to read about the shift theorem.