Calculate a signal of brightness

Question

I am new to signal processing. Currently, I am reading a paper regarding musical expression. The paper calculates brightness, and I want to reproduce their way of computing it. I was wondering if there's any way to calculate harmonic series from audio data, because I want to calculate the Brightness.

This is from a paper talking about a model called [MIDI-DDSP][2]. Here is the section I can't understand very well.

Brightness is defined as the spectral centroid (in bin numbers) of the harmonic distribution, where $h_k(i)$ represents the $k$-th bin of the harmonic distribution, $h(τ)$ in the $i$-th time-step, and we use $|h|$ to refer to the number of bins in the harmonic distribution used by the DDSP module (we use $|h| = 60$, see Appendix B.2).

I am thinking of two ways. The first way is to use spectral_centroid from Librosa.

The second way is to compute the frequency, and harmonic distribution. For example, the frequency of A4 note is 440, and I want the sum of (440*1 + 880*2 + ..) in A4 note.

However, I'm uncertain about the first way. And for the second way, is it correct or not? Any recommendation as to what tools I can use to calculate it?

Answer 1

The paper in question (which you forgot to link to, by the way), has an associated Github. I suggest you start there.

Answer 2

This definition of brightness of timbre in a musical note, and the use of the term spectral centroid was first suggested by the late James Beauchamp, in my recollection. If your musical tone (think of the "tone" as being a slice of the musical note at approximately some time $t_0\ge0$ after the note onset) is:

$$\begin{align} x(t) &= x(t + \tfrac{1}{f_0}) \qquad \qquad & \forall t \approx t_0 \\ \\ &= \sum\limits_{k=-K}^{K} c_k \, e^{j 2 \pi k f_0 t} \qquad \qquad &(c_{-k} = c_k^* \quad c_0 = 0) \\ \\ &= \sum\limits_{n=1}^{K} 2|c_k| \, \cos(2 \pi k f_0 t + \phi_k) \\ \end{align}$$

where

$$ c_k = f_0 \int\limits_{t_0}^{t_0+1/f_0} x(t) e^{-j 2 \pi k f_0 t} \ \mathrm{d}t $$

$$c_k \triangleq |c_k|\, e^{j \phi_k} \quad \text{and} \quad \phi_k \triangleq \arg\{c_k\}$$

Then the power of the tone is

$$ \overline{x^2(t)} = \sum\limits_{k=1}^{K} 2|c_k|^2 $$

and the brightness of the tone is

$$ \frac{\sum\limits_{k=1}^{K} k |c_k|^2}{\sum\limits_{k=1}^{K} |c_k|^2} $$

Some other folks have defined the brightness to be:

$$ \frac{\sum\limits_{k=1}^{K} k^2 |c_k|^2}{\sum\limits_{k=1}^{K} |c_k|^2} $$

Still other used this definition for brightness:

$$ \frac{\sum\limits_{k=1}^{K} k |c_k|}{\sum\limits_{k=1}^{N} |c_k|} $$

It turns out that the definition in the middle is the easiest to implement. But, given these definitions, none of these definitions are functions of the fundamental frequency $f_0$. So, either to extract the $c_k$ coefficients or, if you're using the simplest definition (with $k^2$) and a simple high-pass filter for a differentiator, you need to compute the fundamental frequency and use that in the calculation.

The first and second definition will be easiest to do. We can talk about it, but first I want to get a feel for which way the OP wants to take it.