1
$\begingroup$

Sampling theorem tells us how we can sample a continuous signal such that we can reconstruct it without any errors, or without losing information.

Quantization, on the other hand always results in quantization errors. Why can't we formulate a theorem to enable quantization without errors? I don't see why amplitude and time can't be interchanged, since both are continuous.

I have asked a closely related, but different question on Mathematics Stack Exchange: Can sampling theorem for a real function f be applied for both x and f(x)?

Improve this question
$\endgroup$
7
  • $\begingroup$ if the set of amplitudes is finite, just like a band-limited signal can be presented by a finite set of sampling values. If you can find a useful transform, which should be linear and invertible in most cases IMHO, like Fourier transform, then yes you have your theorem. To the best of my knowledge, we don't have such transform (yet?). $\endgroup$
    – AlexTP
    Jul 5, 2021 at 9:09
  • $\begingroup$ The reason is that quantization is lossy. If you sample and then quantize the time domain, there will be loss of information too -- in consequence, swapping time and amplitude won't work. $\endgroup$
    – MBaz
    Jul 5, 2021 at 13:32
  • $\begingroup$ @MBaz sampling is done ideally at precise intervals of time. But in reality you can't sample so precisely, there will always be timing errors. As AlexTP says in his comment, swapping time and amplitude is indeed possible if the mapping between time and amplitude is bijective $\endgroup$
    – Shashank V M
    Jul 5, 2021 at 13:40
  • 1
    $\begingroup$ @ShashankVM yes the swapping is possible but be careful that such a bijection implies that (every) signal has strictly different amplitude at different sampling time. To the best of my knowledge, we don't treat that kind of signal in practice. $\endgroup$
    – AlexTP
    Jul 5, 2021 at 14:27
  • 1
    $\begingroup$ Yes, there is! Unfortunately I don't have time to write an answer these days but, yes, there is an analogous theorem for the probability distribution of a quantized random signal. If I find the time, I'll at list point you to the literature. $\endgroup$
    – Massimo Ortolano
    Jul 5, 2021 at 18:49

4 Answers 4

Reset to default
3
$\begingroup$

We have a coherent and consistent theory of sampling that proves that band-limited signals can be sampled without loss of information.

We have a coherent and consistent theory of quantization that proves that quantization always creates an error

The question of "why" is meaningless. Things are what they are.

In any real world application you need to reduce the theory to practice. That means that sampling still creates error, since band-limited signals don't actual exist and that quantization can always be made "good enough" since there is always some noise floor somewhere.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ Sorry, I didn't phrase this well and this would be a topic of a lengthy philosophical discussion. What I mean is that the idea of "how things SHOULD work" get often in the way of understanding "how things actually work". Quantum Mechanics borders on the absurd but it's nonetheless a wildly successful theory. If you are interested in this sort of thing: amazon.de/-/en/Michael-Strevens/dp/1631491377 $\endgroup$
    – Hilmar
    Jul 5, 2021 at 13:41
  • $\begingroup$ Ok, I get your point. $\endgroup$
    – Shashank V M
    Jul 5, 2021 at 13:42
1
$\begingroup$

Sampling theorem tells us we can sample only a band-limited signal. This means, the signal should have a maximum frequency with respect to time.

If a signal has a maximum frequency with respect to amplitude, we can apply sampling theorem to the amplitude as well.

In the sampling theorem, it is not necessary for the x-axis to be time for it to be applied. It is just a mathematical theorem.

The original version of Sampling Theorem:

If a function $x(t)$ contains no frequencies higher than $B$ hertz, it is completely determined by uniform samples taken at a rate $f_s$, where $f_s >=2B$ seconds apart.

A standard signal in this case is considered a function of time. We can also consider time as a function of amplitude of a signal. $t(x) = sin(x)$, where $x = sin^{-1}(t)$. If $sin(x)$ has a maximum frequency $B$, there won't be any quantization error if quantization is done uniformly spaced at intervals of $1/2B$ or lesser.

As user AlexTP says in their comment: In quantization, the mapping from time to amplitude is injective and, therefore, a function; whereas the inverse mapping is not (one time can only have one amplitude). If you assume the inverse mapping to be also injective to make it a function, the mapping is bijective; so "amplitude" and "time" are interchangeable.

So the answer is if the relationship between time and amplitude of a signal is bijective, then we can have an analogous sampling theorem for quantization.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This "If a signal has a maximum frequency with respect to amplitude," is very important. Please formulate it mathematically and put it in your question. $\endgroup$
    – AlexTP
    Jul 5, 2021 at 9:11
  • $\begingroup$ @AlexTP I have added it mathematically with the help of an example. Please have a look $\endgroup$
    – Shashank V M
    Jul 5, 2021 at 12:35
  • 1
    $\begingroup$ For you update: in the context of the operation of interest, i.e. quantization, the mapping from time to amplitude is injective and, therefore, a function ; whereas the inverse mapping is not (one time can only have one amplitude). If you assume the inverse mapping to be also injective to make it a function, the mapping is bijective; so "amplitude" and "time" are just two interchangeable name and yes, you have the theorem. Unfortunately, these two sets "amplitude" and "time" are not commonly used in practice. $\endgroup$
    – AlexTP
    Jul 5, 2021 at 13:22
1
$\begingroup$

It may help to understand that a critical difference is time is not the independent variable when we are sampling. When we sample, we quantize time and at that specific sample location we then determine the dependent variable (which we can the quantize or not as a different process, sampling in time is independent of sampling in magnitude).

Specific to the time sampling: Given an arbitrary independent signal that is being sampled, observe how the error is a random white noise process with a uniform distribution between two quantization levels (plus or minus half a quantization level or from 0 to a quantization level depending on if we are rounding or truncating). Given it is random and independent from sample to sample (white noise), we are unable to reconstruct the error without knowing further information about the signal itself, but we are able to represent the error reasonably well as a random process.

The time error follows a similar distribution, and would be of use when our dependent variable is time. If we are interested in a precise time, but can only sample it over certain intervals, then we do have time error from which we can determine the precise time it we had precise noise-free amplitude samples. We do this in timing recovery algorithms with variable delay elements and effective resampling (even when we don't actually resample) to resolve time to finer precisions of interest. Sampling error also follows a uniform distribution as a random process and as such the RMS (Root Mean Square) jitter would be $T_s/\sqrt{12}$ (consistent with the standard deviation of any uniform distribution).

I’ve used this relationship for example in radar range resolution given the sampling rate of the system establishing the time resolution, and predicting the actual arrival time from the time in which a threshold detector running off of the local sampling clock actually triggers. This would be an example where time is the dependent variable and showing how we have the same treatment of the error.

Improve this answer
$\endgroup$
5
  • $\begingroup$ The answer is when you are concerned with precise time that you can extract it from your samples and that this is done routinely $\endgroup$
    – Dan Boschen
    Jul 5, 2021 at 13:14
  • $\begingroup$ Meaning you can interchange them; you are implying you can’t. $\endgroup$
    – Dan Boschen
    Jul 5, 2021 at 13:15
  • $\begingroup$ Yes theoretically you can, make the units of amplitude the dependent variable and time the dependent variable and you can apply the sampling theorem to that. There is nothing special or exclusive about the units- the important thing is to identify which is dependent which is the point of my answer. $\endgroup$
    – Dan Boschen
    Jul 5, 2021 at 13:19
  • $\begingroup$ My explanation of the random process that results didn’t answer it for you, or maybe you don’t see they yet and want more detail as to why it would be a random process and as such each individual error sample is not predictable / removable? $\endgroup$
    – Dan Boschen
    Jul 5, 2021 at 13:26
  • $\begingroup$ What about time error are you referring to? Are you satisfied with the response that without prior info about the signal itself that the amplitude process is a random process; I could add a white noise process meaning every sample is independent of the next. I think that is the salient point to your question so please let me know if that is not clear or debated and I can add more detail to that point. (Or if you are unfamiliar with random processes overall which would make that more confusing then) $\endgroup$
    – Dan Boschen
    Jul 5, 2021 at 13:37
1
$\begingroup$

A perfectly band-limited signal must have infinite support, thus must be infinitely long. A signal that can be sampled without error at any sample spacing of frequency below Nyquist, must have a value that is an exact integer multiple of your quantization spacing.

So I think the constraint required for both of your reconstruction theorems to hold only includes infinite DC signals of an exact quantization integer multiple in value. e.g. a constant.

Not very useful.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.