When I started to write a software for chromatogram processing I naively thought I'd find some existing algorithms that do all the peak detection automatically and there won't be a need for any manual curation whatsoever. But it appeared that none of the major software in this field works this way. Users have to manually set up parameters (peak widths, thresholds) which work on some chromatograms but not others. So it's either an endless tuning of parameters or manual correction.

My question is philosophical: what keeps this problem unsolved? Do other fields (astronomy, particle physics) have similar problems interpreting measurements automatically? Can other approaches except DSP (e.g. Machine Learning) be useful here?


This is a problem we are also currently working on (added two recently published papers below), because we found it unresolved yet, and high-throughput experiments nowadays require more automated high-throughput analyses.

A first set of causes are DSP-like disturbances on the "interesting peak data": non-linearity (like saturation), added noise (with unknown distributions) and baselines (with unknown regularity). They can related to solute or matrix effects, type of chromatography (liquid, gas), columns, detectors, the complexity of the mixture, etc. We can include more or less visible processing performed inside the measurement tools: sampling (how any points per peaks?), smoothing, normalizations.

In gas chromatography there are models like Gaussian shapes, that can be explained by physical models (plates). However, separated peaks can exhibit distorsions such as fronting and tailing, or other asymmetries, that can vary throughout the chromatogram in uneven manners.

Even with "nicely"-sampled discrete Gaussians, very close peaks may be difficult to separate or distinguish (shouldering effects). More globally, deconvolution of sums of Gaussians remains difficult, because the direct problem is ill-posed: there are many ways to fit a single Gaussian with one of several Gaussians, and algorithms require additional optimization terms (constraints, regularization), that come with tuning parameters when combined with the data fidelity term (often a least-squares, but other choices are interesting).

As far as I know, such issues exists in other analytical chemistry modalities: NMR, infrared, mass spectrometry, Raman etc., each with their own issues. Variable peak shifting related to molecule interactions in NMR is a challenge for subsequent (blind) source separation algorithms.

There are works on using more machine learning techniques, yet it is likely that using physical or DSP models would help them.

A possible starting point on the difficulties of fitting: Dependence of Chromatogram Peak Areas Obtained by Curve-Fitting on the Choice of Peak Shape Function, 1997

We recently considered a couple of methods towards reconstruction or recovery of physico-chemical signals. They definitely are not the end of the game:

SPOQ ℓp-Over-ℓq Regularization for Sparse Signal Recovery applied to Mass Spectrometry, IEEE Transactions on Signal Processing, 2020 (Matlab code SPOQ toolbox)

Underdetermined or ill-posed inverse problems require additional information for \ldd{d} sound solutions with tractable optimization algorithms. Sparsity yields consequent heuristics to that matter, with numerous applications in signal restoration, image recovery, or machine learning. Since the ℓ0 count measure is barely tractable, many statistical or learning approaches have invested in computable proxies, such as the ℓ1 norm. However, the latter does not exhibit the desirable property of scale invariance for sparse data. Extending the SOOT Euclidean/Taxicab ℓ1-over-ℓ2 norm-ratio initially introduced for blind deconvolution, we propose SPOQ, a family of smoothed (approximately) scale-invariant penalty functions. It consists of a Lipschitz-differentiable surrogate for ℓp-over-ℓq quasi-norm/norm ratios with p∈]0,2[ and q≥2. This surrogate is embedded into a novel majorize-minimize trust-region approach, generalizing the variable metric forward-backward algorithm. For naturally sparse mass-spectrometry signals, we show that SPOQ significantly outperforms ℓ0, ℓ1, Cauchy, Welsch, SCAD and Celo penalties on several performance measures. Guidelines on SPOQ hyperparameters tuning are also provided, suggesting simple data-driven choices.

Sparse Signal Reconstruction for Nonlinear Models via Piecewise Rational Optimization, Signal Processing, 2021

We propose a method to reconstruct sparse signals degraded by a nonlinear distortion and acquired at a limited sampling rate. Our method formulates the reconstruction problem as a nonconvex minimization of the sum of a data fitting term and a penalization term. In contrast with most previous works which settle for approximated local solutions, we seek for a global solution to the obtained challenging nonconvex problem. Our global approach relies on the so-called Lasserre relaxation of polynomial optimization. We here specifically include in our approach the case of piecewise rational functions, which makes it possible to address a wide class of nonconvex exact and continuous relaxations of the ℓ0 penalization function. Additionally, we study the complexity of the optimization problem. It is shown how to use the structure of the problem to lighten the computational burden efficiently. Finally, numerical simulations illustrate the benefits of our method in terms of both global optimality and signal reconstruction.

  • 1
    $\begingroup$ yeah, Gaussian Mixture Models seem mean to me – simply because from an information theoretical point of view, for any differential entropy of a continuous source, the Gaussian source has the highest entropy, i.e. the most uncertainty about the outcome. $\endgroup$ – Marcus Müller Nov 7 '20 at 10:52
  • 1
    $\begingroup$ One idea I had in the past for the "close-by (relative to their variance) Gaussians" is to optimizes a metric that works both in the original and a Fourier space of the data – simply because Gaussians remain Gaussians, but large variances become low ones – and then be a bit nonlinear in weighting, so that things that are sharp in either domain, but still "exist" as Gaussians in the other, are overpronounced. But then I never came around to implementing/error-bounding it. $\endgroup$ – Marcus Müller Nov 7 '20 at 10:52
  • $\begingroup$ I am interested very much in your idea. If you decide to make it public one day, I would be glad to know. Progresses in optimization allow to better optimize non-classical metrics. Additionally, I am trying to model baselines as a sum of a few overlapping signed large gaussians, whereas peaks are numerous (but sparse) fine Gaussians, almost separated. There should be space between $\endgroup$ – Laurent Duval Nov 7 '20 at 11:30
  • $\begingroup$ The link as been corrected $\endgroup$ – Laurent Duval Nov 11 '20 at 3:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.