# Questions tagged [quantization]

Quantization, in mathematics and digital signal processing, is the process of mapping input values from a large set (often a continuous set) to output values in a (countable) smaller set, often with a finite number of elements. Rounding and truncation are typical examples of quantization processes.

28 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
96 views

### Distortion-Rate function for standard Gaussian signals quantized with a uniform alphabet

Let's say $X \sim N(0, 1)$ is a standard Gaussian input signal and $Y \sim U\{y_1, y_2, \cdots, y_{16}\}$ follows a discrete uniform distribution (in fact, $\{y_1, y_2, \cdots, y_{16}\}$ is a ...
• 125
99 views

### Modelling of quantization noise with logarithmic steps

I have to modell the quantization noise of ADC. I only know that the maximum relative error is 1 dB. Since this is logarithmic, I find it hard to formulate an appropriate model for the quantization ...
• 21
599 views

### Quantization step-size for a given signal based on its PDF

I am familiar with the principles of midtread and midrise quantizer. However, I have difficulties determining the step size where it hasn't been explicitly given. For example, the following ...
• 467
1 vote
82 views

### Detect digitally time stretched audio made back to tape

Does anyone know how to detect and quantify a digitally time stretched audio signal when made back to tape?
• 11
1 vote
29 views

### Novice question about mid-rise quantization

Accoarding to wikipage, the function for quantization has the ceiling of x/Ds. Is this the ceiling of something multiplied by Ds?
• 111
1 vote
123 views

### What is wrong with my 3D DCT quantization?

I'm implementing 3D DCT quantization as in Quantization of 3D-DCT Coefficients and Scan Order for Video Compression (sorry about the paywall). They propose the following equation for generating the ...
• 248
1 vote
310 views

### Which type of quantization is better?

If a signal's amplitude has a uniform PDF, which is better in the sense of producing a smaller quantization error: optimal scalar quantization or quantization with compressor (non-uniform quantization)...
• 31
1 vote
210 views

### Filter output quantization - does this limit effective coefficient quantization?

I am implementing a filter in hardware, and the intended maximum data input is 16-bit. The question is, if the output is to be re-quantized to 16-bit, does this mean certain coefficients are ...
1 vote
72 views

### Dealing with quantization level limiting dsp

There is a analog signal in the form as below: $$f(t)=A \sin(\omega(t) t+\phi)+B \sin(\omega(t-t_1) t+\phi_1)$$ Where $$B<<A$$ We are going to calculate $\phi, \phi_1, t_1$,using dsp. But ...
• 21
1 vote
153 views

### Question on PCM Sampling and Quantization order

I had a doubt in my mind regarding the orders of sampling and quantization in PCM. What is the impact if the order is reversed, that is, the continuous time signal is first quantized and then sampled? ...
• 367
34 views

### Calculation of Compression Ratio after DCT

I am seeking information about the compression ratios achieved at each stage of DCT-based compression processes. Specifically, I'm interested in understanding the compression ratio immediately after ...
45 views

### How does quantization do to tell the difference between one sampling point and one other?

The signal quantization consists in assigning to the different values of the sampled signal a value among the range of the $2^p$ values allowed by the $p$ bits used. As the different values of the ...
• 101
66 views

### Filter away INT_MIN when doing DSP since -INT_MIN is undefined(?)

My problem is that I have a mic that delivers 24 bits 2's complement signed. There is nothing in that data sheet that hints at it not also delivering INT_MIN. The 8 ...
43 views

### How to design an optimal LLR Quantization of LLR for LDPC Decoding under AWGN channel using BPSK(Known quantization boundaries)?

Recently I met an optimazition problem, i.e., finding an optimal quantizer which maps floating point LLRs to integers of range $-11,\ldots +11$, where the quantizer boundaries have been calculated out....
• 141
42 views

### Cross-quantilogram method

What is this method in general? What is the main difference between the quantile cross-spectrum (QS) approach developed by Baruník and Kley (2019) and the cross-correlation function (QS) developed by ...
23 views

### Approximating variable-length coding by "aggregating" fixed-length coding

Consider the problem where I want to quantize a univariate standard Gaussian with two bits, and the objective is to minimize MSE reconstruction loss. According to these slides, the Lloyd algorithm and ...
• 125
137 views

### AWGN model of Quantization Noise

[Edit answer moved from OP edit to "Answers" as per comment suggestion] I'm trying to create a simple model of a signal chain (image sensor) that includes an ADC but I'm failing to observe ...
• 51
82 views

### What order to apply noise shaping and anti-aliasing to an idealized synth

I am trying to write a small synth code that goes something like this, I have an abstract symbolic sound representation, sine/square/saw waves. To generate final sound and make it sound good I thought ...
• 101
70 views

### How to add quantization noise (SQNR) to thermal noise (SNR) in a total value?

How can I add the BAQ (Block Adaptive Quantizer) compression noise to the input SNR? In the end is a quantization noise to be added to a thermal noise. I got the following formulation, but I dont ...
• 35
151 views

### Scalar vs Vector signal Quantization

I am studying about scalar vs vector quantization, and I have an assignment to implement (in MATLAB) a scalar quantizer , using the Lloyd-Max algorithm, and a vector quantizer via k-means clustering. ...
52 views

### How to find the minimum no of bits required to express the filter coefficients in fixed point format from Histogram plot?

I have designed a filter in MATLAB and now I have to convert the filter coefficient in fixed point format from floating point format. For that my professor asked me to find histogram first and then ...
520 views

### SNR of μ-law companding

I know the snr after μ-law compression is $\text{SNR}\approx\frac{3L^2}{[\ln(1+ \mu)]^2}$ when $\mu \gg \frac{\max_t m(t)}{\text{rms}(m(t))}$, where $m(t)$ is the message signal, and $L$ the number ...
67 views

### Signal to Quantization noise problem

a full scale signal of bandwidth 5 khz is sampled by an 10-bit ADC at a sampling rate of 2 Msa/sec calculate the Signal to Quantization noise of the resulting DT signal repeat for a 14-bit ADC at ...
31 views

### Quantization of Image which have taken from partially less dark

I have picture with high resolution with 20M memory space and this image has been taken from the standing away from the window in room which has significantly less light as compared to outside. Image ...
• 101
68 views

### A/D converter quantization noise

I read that in A/D converter we usually use rounding or two's complement truncation and not the sign magnitude or 1's complement truncation as in this case the quantization error is correlated to the ...
• 135
1k views

### How to Prevent overflow/saturation in fixed point implementation of Fast Data Projection Algorithm

I am trying to implement an algorithm in real-time on a Fixed point DSP (The Blackfin from Analog Devices). The algorithm does a lot of stuff, but in the middle it performs an algorithm called "Fast ...
• 1,241
156 views

### Harmonics in Quantal Histogram

I'm studying what I believe to be a system that produces a (noisy) quantal response (i.e. there should be a minimum increment of response). To do this, I've plotted a histogram of some ~6000 amplitude ...
• 1