I have, in the past, used delay modulation on arbitrary audio signals. but the modulation waveform itself was not arbitrary, but was a sinusoid. but we could speed this modulation oscillator up to an audio rate.
Now that I think of it, I tried hooking that modulation oscillator up to a pitch detector. This is a quarter century ago, so I don't remember everything.
I s'pose you could derive that delay modulation signal from the input and likely LPF it before applying it to the instantaneous delay of the signal going out.