2
$\begingroup$

I'm working on a dataset of spectroscopies and i'm classifying them by transforming the 1D signal into a 2D one by the ssqueezepy library.

For instance, consider to have a 1D signal and the corresponding 2D image, generated by executing the function:

Tx, _ = ssq_cwt(input_1D_signal, wavelet="gmw", nv = 64, scales=0.1)

From the library above. The study I performed highlights the areas of the cwt image that are most important for classification. Do you know if there is a way to find the corresponding information in the original 1D signal? I'm using the inverse mapping in the library, issq_cwt(Tx), but I don't know how to find the corresponing highlighted informations.

Unfortunately I'm nearing a deadline, so I can't immerse myself much in the literature behind it, so I hope someone who already knows the synchrosquezing procedure can help me.

Edit: Looking inside the library code is explained that the inverse function above is from the eq.15 of this paper.

There is a way to relate the math procedure done on the image of the transformed signal directly on the heatmap image? I'm trying to brute-pass the grayscale heatmap to the issq_cwt() function to obtain a result(which even seems acceptable), is that correct?

$\endgroup$
3
  • $\begingroup$ Can you please explain, how the exponential frequencies are confined by two straight lines? is the y-axis log-scaled? $\endgroup$ Jul 11, 2022 at 15:55
  • $\begingroup$ @user3708408 Welcome to SE.SP! Please do not post a new question as an answer to another question. If you have a new question, please ask it. Please do not expect answers to new questions in the comments section. $\endgroup$
    – Peter K.
    Jul 11, 2022 at 16:15
  • 1
    $\begingroup$ @user3708408 Yes, and it's not just the plot but the transform itself. Some learning resources (see 1). $\endgroup$ Jul 12, 2022 at 7:45

2 Answers 2

1
$\begingroup$

Any time-frequency representation with hop_size=1 is subject to the one-integral inverse, either directly or with a normalization step. This means we recover x[0] via sum(Sx[start:end, 0]), where Sx.shape = (freqs, time), and start:end is the region of interest.

ssqueezepy.cwt uses hop_size=1 and is L1-normalized by default, meaning it's subject to direct inversion (within a constant scaling factor taken care of by icwt). The README shows an example in action; an exponential chirp contaminated by severe noise

enter image description here

is recovered by setting start, end for each timestep such that they form the two red lines shown (which are implicitly shaded in-between). That's the region that's inverted as the orange waveform in bottom right.

The practical challenge is converting such red bands into code, as indices. I'm not aware of any existing tool that does this, so I started to write one: it saves the CWT/SSQ as an image, then you draw on it with a red line/curve using some tool like MS.Paint, then the edited image is loaded and indices are automatically extracted. Unfinished code here.

Extracting such indices automatically is much more challenging, and is the subject of ridge extraction. ssqueezepy has a fairly good algorithm, but it's not entirely reliable, nor do I know if there's any algo that is.

$\endgroup$
0
$\begingroup$

"Synchrosqueezing". Never heard of that before, but it looks suspiciously similar to something I devised years ago. In fact, looking at a description, it just so happens to be the same thing.

You're looking at the actual scalogram of the signal. Note: "scalogram", not "spectrogram". A spectrogram takes place in the time-frequency plane. The horizontal coordinate is the time, the vertical coordinate is the frequency, with equal spacing between 0 Hz, 100 Hz, 200 Hz, 300 Hz and so on.

A scalogram is in the time-scale plane - which is the distinguishing feature of time-scale transforms like the wavelet, S and Q transforms. Its main difference is that the "scale" is logarithmic. What's called "scale" in wavelet analysis is, essentially, the reciprocal of frequency, so that a scalogram is - equally so - the depiction of frequency on a logarithmic scale. That means there is equal spacing between octaves, and the bottom of the graph is not 0 Hz, but the base frequency included in the analysis, whatever that may be. Logarithm of scale is negative logarithm of frequency, so the diagrams would be upside-down.

"synchrosqueezing" is one and the same as is called "frequency reassignment" for spectrograms. A bona fide spectrogram (or scalogram) would show not just the intensity (e.g. by color coding or brightness) but also the phase. The way I do it is to show the amplitude by brightness and the phase by color. In contrast, the spectograms (and scalograms) most people think of or are familiar with show only the amplitude. The spectrogram in Audacity, for instance, shows only amplitude, not phase.

The proper way to handle phase, short of outright depicting it, is to use it by reassigning frequencies. The inverse process is that of trying to re-insert phase into an amplitude only spectrogram or scalogram, which entails another range of methods; the earliest of which was called the "vocoder".

Um ... the NSA already has secret patents on some of the methods related to frequency relocation; except some are no longer secret. The one that slipped out is valid until the late 2020's but works with relocation and spectrograms, rather than scalograms. As far as I'm aware, the other patents are still secret (officially speaking). Their secret synchrosqueezing-related patents are probably still under wraps.

Here's what a scalogram looks like without relocation, but with the phase color-coded ... in 1/4 real-time.

https://www.youtube.com/watch?v=6orozX1GD1w

The largest scale (and smallest frequency) is at the top, the smallest scale (and largest frequency) is at the bottom.

If you pay careful attention to the depiction, you will notice that there are two naturally occurring components. The rate of variation of the phase is the same, for a given component, irrespective of its vertical location on the diagram. That shows that we're picking up the same natural frequency for a single component, via the different channels or "voices" that comprise each row in the diagram. So, when it's relocated, all of that component is added up and relocated at the spot corresponding to its natural frequency.

What's its natural frequency? The rate at which the phase cycles through. The diagram is 1/4 second across, and you can see it cycling through the red, yellow, green, cyan, blue, magenta, back to red about 15 times across the diagram. So, it's about 60 Hz. According to the video description, the top is MIDI -37, the bottom is at MIDI 117. The sound is at 1/4 speed.

You'll see in the diagram a second component, mostly on the bottom, with much more tightly-spaced cycles ... so much so that it's virtually whited out. That's another sound component.

The transform used is not bona fide wavelet, since it used a different transform windowing function for the reverse direction than for the forward direction. The wavelet transform (unnecessarily) uses the same windowing function for both the forward and reverse transform. Here, the reverse transform is to simply add up the individual components from top to bottom. (The S-transform has a similar prescription for the inverse transform, but with a slight difference that complicates the picture.)

You could, in fact, play each one out separately and get the respective sound component. But the filtering used was not clean, so the high frequency component still carries a bit of the low frequency sound in it. That shows up as the "wigglies" in the higher frequency components.

This is an example with relocation.

https://www.youtube.com/watch?v=itUSUau6DJM

Finally, this is a full run with the heartbeat House Music sound and voice ... that was put up just recently - a replay of the last segment of the one I just showed up above.

https://www.youtube.com/watch?v=nd2J4xTrSHQ

You can literally see the individual syllables of the preacher's speech (and that of the audience members)! And, you see the tone and drum beat. "scale" has been switched to "frequency", so the high end is high frequency, and the low end is low frequency.

This one pretty closely matches the kind of chart you have. But the time resolution is too coarse to see the color-coding of the phase (plus, when YouTube converts the video, is smeared out what coloring was visible).

$\endgroup$
5
  • $\begingroup$ Hmm, "same as synchrosqueezing" is a strong claim. What distinguishes SSQ from other reassignment methods is its reassigning criterion and operator, being respectively the nearest phase derivative and addition. This confers advantages other methods don't enjoy; I provide an overview here. If yours is different, consider making a Q&A. $\endgroup$ Jul 12, 2022 at 7:45
  • $\begingroup$ No, it's just a simple fact: they are the same thing. There are a wide range of methods for carrying out the process; corresponding to different method of discretization. But it is all classified under what is known as "relocation". Whether it is done with spectrograms or scalograms is immaterial; in all cases, you're using the same formulae. The "differences" you're alluding to are bit different methods for digitizing and discretizing these formula: meant to produce the same outcome. So, you're talking about distinctions without a relevant difference. $\endgroup$
    – NinjaDarth
    Jul 12, 2022 at 8:16
  • $\begingroup$ I'm afraid that's wrong. The methods differ drastically, and aren't merely alternate discretizations. Many lose inversion, for example, unlike SSQ. $\endgroup$ Jul 12, 2022 at 8:33
  • $\begingroup$ I'm afraid "that's wrong" is wrong - by your own admission. You actually called it a reassignment method in your own description (which is what I said and what you're responding to). It is simply another reassignment method. I've reviewed your material. There is nothing new or particularly significant in it. So, I tell you what I'm going to do. Since you're making an issue of it; then take the 3 videos I cited, grab the sounds, run your analysis on them and compare results. In the meanwhile, I will post an "issue" on your GitHub page with a description of the transform therein for you to try. $\endgroup$
    – NinjaDarth
    Jul 13, 2022 at 20:56
  • $\begingroup$ CWT and STFT are time-frequency methods, but we don't say "CWT is same as STFT". If fixing quadratic interference and loss of inversion isn't significant, to each their own, but you won't find many in agreement. $\endgroup$ Jul 15, 2022 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.