0
$\begingroup$

I am new to signal processing. Currently, I am reading a paper regarding musical expression. The paper calculates brightness, and I want to reproduce their way of computing it. I was wondering if there's any way to calculate harmonic series from audio data, because I want to calculate the Brightness.

This is from a paper talking about a model called [MIDI-DDSP][2]. Here is the section I can't understand very well.

Brightness is defined as the spectral centroid (in bin numbers) of the harmonic distribution, where $h_k(i)$ represents the $k$-th bin of the harmonic distribution, $h(τ)$ in the $i$-th time-step, and we use $|h|$ to refer to the number of bins in the harmonic distribution used by the DDSP module (we use $|h| = 60$, see Appendix B.2).

I am thinking of two ways. The first way is to use spectral_centroid from Librosa.

The second way is to compute the frequency, and harmonic distribution. For example, the frequency of A4 note is 440, and I want the sum of (440*1 + 880*2 + ..) in A4 note.

However, I'm uncertain about the first way. And for the second way, is it correct or not? Any recommendation as to what tools I can use to calculate it?

$\endgroup$
2
  • $\begingroup$ It looks like you have intended to include an image, but there is not link included. $\endgroup$ Commented Aug 21, 2022 at 10:50
  • $\begingroup$ Oh, that can be deleted. $\endgroup$
    – Megan
    Commented Aug 21, 2022 at 13:13

2 Answers 2

1
$\begingroup$

The paper in question (which you forgot to link to, by the way), has an associated Github. I suggest you start there.

$\endgroup$
2
  • $\begingroup$ Yeah, I've already read it. But the codes are so hard to understand. It's unclear $\endgroup$
    – Megan
    Commented Aug 21, 2022 at 13:45
  • 4
    $\begingroup$ @Megan then you'll need to try and ask precise questions. If you know something is big, "can you explain this big thing to me, I don't tell you where I'm stuck at understanding it" is not a strategy that leads to answers. $\endgroup$ Commented Sep 4, 2022 at 9:29
0
$\begingroup$

This definition of brightness of timbre in a musical note, and the use of the term spectral centroid was first suggested by the late James Beauchamp, in my recollection. If your musical tone (think of the "tone" as being a slice of the musical note at approximately some time $t_0\ge0$ after the note onset) is:

$$\begin{align} x(t) &= x(t + \tfrac{1}{f_0}) \qquad \qquad & \forall t \approx t_0 \\ \\ &= \sum\limits_{k=-K}^{K} c_k \, e^{j 2 \pi k f_0 t} \qquad \qquad &(c_{-k} = c_k^* \quad c_0 = 0) \\ \\ &= \sum\limits_{n=1}^{K} 2|c_k| \, \cos(2 \pi k f_0 t + \phi_k) \\ \end{align}$$

where

$$ c_k = f_0 \int\limits_{t_0}^{t_0+1/f_0} x(t) e^{-j 2 \pi k f_0 t} \ \mathrm{d}t $$

$$c_k \triangleq |c_k|\, e^{j \phi_k} \quad \text{and} \quad \phi_k \triangleq \arg\{c_k\}$$

Then the power of the tone is

$$ \overline{x^2(t)} = \sum\limits_{k=1}^{K} 2|c_k|^2 $$

and the brightness of the tone is

$$ \frac{\sum\limits_{k=1}^{K} k |c_k|^2}{\sum\limits_{k=1}^{K} |c_k|^2} $$

Some other folks have defined the brightness to be:

$$ \frac{\sum\limits_{k=1}^{K} k^2 |c_k|^2}{\sum\limits_{k=1}^{K} |c_k|^2} $$

Still other used this definition for brightness:

$$ \frac{\sum\limits_{k=1}^{K} k |c_k|}{\sum\limits_{k=1}^{N} |c_k|} $$

It turns out that the definition in the middle is the easiest to implement. But, given these definitions, none of these definitions are functions of the fundamental frequency $f_0$. So, either to extract the $c_k$ coefficients or, if you're using the simplest definition (with $k^2$) and a simple high-pass filter for a differentiator, you need to compute the fundamental frequency and use that in the calculation.

The first and second definition will be easiest to do. We can talk about it, but first I want to get a feel for which way the OP wants to take it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.