Introducing Feedback to Miller Puckette's Time-Domain Pitch Shifting Algorithm without Discontinuities

Question

I've implemented Miller Puckette's time-domain pitch shifting algorithm (http://msp.ucsd.edu/techniques/v0.11/book-html/node125.html) in PureData's help files in C++ (my code here: https://github.com/aparks5/synthcastle/blob/main/src/pitchshift_timedomain.cpp)

I've been playing around with it, and noted that introducing feedback into the delay line produces some interesting effects. I could describe it is a "bloom" but it is probably more clearly described as a pitch-shifter with (exponentially?) further increasing pitch over time.

However, because of the assumptions made about the phase of the envelope and the phase of the unipolar sawtooth which modulates the delay time, when feedback is introduced, the assumptions about phase that allow this splicing to occur cleanly without discontinuities no longer holds.

How can I introduce feedback into a pitch-shifting delay line without introducing further clicks?

I don't expect it's going to sound "clean" without Phase Vocoding, I just simply want a better idea of how I can modify the envelope to compensate for the feedback path.

Anecdotally, I think this effect must be possible because it was implemented on the Boss RPS-10 Pitch-Shifter Delay, and I have to imagine such an effect worked in the time domain.

Here is my code pasted below

#include "pitchshift_timedomain.h"

#include "gain.h"

PitchShift::PitchShift(size_t fs)
    : Module(fs)
    , m_delay(fs, 1.0f)
    , m_delayInv(fs, 1.f)
    , m_saw(fs)
    , m_shiftSemitones(7)
{ 
    m_delay.update(0.f, 0.0f);
    m_delayInv.update(0.f, 0.0f);
    // http://msp.ucsd.edu/techniques/v0.11/book-html/node125.html
    float semi = m_shiftSemitones;
    float temp = expf(semi * 0.05776f) - 1;
    temp = temp / 0.08; // window
    m_saw.freq(temp);
}

float PitchShift::operator()(float in) {

    // http://msp.ucsd.edu/techniques/v0.11/book-html/node125.html
    float delay1Modulation = m_saw();
    // make saw unipolar
    delay1Modulation = (delay1Modulation * 0.5f) + 0.5f;

    // add 90 degree offset, wrap around 1
    float delay2Modulation = 0.5f + delay1Modulation;
    if (delay2Modulation > 1) {
        delay2Modulation -= 1;
    }

    m_delay.update((delay1Modulation * 80) + 67, 0.0f);
    m_delayInv.update((delay2Modulation * 80) + 67, 0.0f);

    float temp = m_delay();
    // restrict cosine from -.25 to .25
    float env = cosf(2*M_PI * ((delay1Modulation - 0.5f) * 0.5f));
    temp *= env;
    
    float tempInv = m_delayInv();
    float envInv = cosf(2*M_PI * ((delay2Modulation - 0.5f) * 0.5f));
    tempInv *= envInv;

    m_delay.write(in);
    m_delayInv.write(in);
    return (temp + tempInv) * 0.707;
}