I am studying the problem of digital symbols detection in ISI channel. In the figure below, the output symbol is represented as a weighted sum of input symbols.

enter image description here

I am trying to use MAP decoder comm.APPDecoder to estimate the transmit symbols. To construct the trellis of such figure, I am using the Matlab function poly2trellis. The function can be used as follows:

trellis = poly2trellis(ConstraintLength,CodeGenerator)

My question is how can I enter (or consider) the real-valued coefficients $\Gamma_{-L_2}, ..., \Gamma_{L_1}$ in the second input argument CodeGenerator. If these coefficients are binary, then it is straightforward to convert this binary sequence to octal. But, I have no clue with real-valued coefficients. Any hints?

  • 1
    $\begingroup$ poly2trellis is used for convolutional encoders which are defined by generators over binary fields. What you have in your figure, with real-valued coefficients, resembles more to a digital filter (more specifically an FIR filter which would have the feedback coefficients a = 1). $\endgroup$
    – SleuthEye
    Jan 10, 2017 at 1:29
  • $\begingroup$ All your edits point to @SlothEye's initial suspicion being true - what you describe is a FIR filter, and the inverse of that isn't APPDecoder, but an equalizer. $\endgroup$ Jan 10, 2017 at 2:10
  • $\begingroup$ Traditional equalization performance is not satisfactory if the ISI is sever. That is why I am looking for the optimal detection using APP decoder. $\endgroup$
    – Noor
    Jan 10, 2017 at 2:22
  • $\begingroup$ But what you describe is not a decoder problem. You first encode, then you apply a constellation and a pulse shape, then the isi channel happens, you then equalize, matched filter, de-constellation-map and then you decode. You get two problems: estimating the channel for the equalizer, and estimating the info bit sequence based on the code bit sequence. Isi doesn't happen to bits - it happens to symbols in their baseband shape. $\endgroup$ Jan 10, 2017 at 2:42

1 Answer 1


As you mentioned, applying linear equalization (e.g. filtering with the inverse of the channel) will not yield the optimal performance. Though, what you are searching is not the comm.APPDecoder (which is solely for convolutional codes, i.e. bits with binary $\Gamma$), but you are looking for maximum likelihood sequence estimation (MLSE).

Here, MATLAB offers the comm.MLSEEqualizer object, which you can e.g. use like this:

Constellation = [1+1j, 1-1j, -1+1j, -1-1j];
Channel = [1 0.7, 0.3, 0.2]';
sigma2 = 0.3;

tx = Constellation(randi(4, [100,1])).';

noise = sqrt(sigma2/2)*complex(randn(size(tx)), randn(size(tx)));
rx = filter(Channel, 1, tx) + noise;

mlse = comm.MLSEEqualizer('Channel',Channel,'Constellation',Constellation);

estimated = step(mlse, rx);

[estimated tx abs(estimated-tx)]

program output (only a few rows of it):

   1.0000 - 1.0000i   1.0000 - 1.0000i        0          
  -1.0000 + 1.0000i  -1.0000 + 1.0000i        0          
   1.0000 + 1.0000i   1.0000 + 1.0000i        0          
   1.0000 + 1.0000i   1.0000 + 1.0000i        0          
   1.0000 + 1.0000i   1.0000 + 1.0000i        0          
  -1.0000 - 1.0000i  -1.0000 - 1.0000i        0          
  -1.0000 - 1.0000i  -1.0000 - 1.0000i        0          
   1.0000 + 1.0000i   1.0000 + 1.0000i        0  

as you see, the MLSE equalizer completely recovers the transmitted sequence.

  • $\begingroup$ This is helpful. Thanks. However, it is not exactly what I am looking for as it does not provide soft outputs (to use with outer channel decoder). If 'comm.APPDecoder' works only with binary $\Gamma$, then the only way is to write my own BCJR or MPA algorithms? Right? $\endgroup$
    – Noor
    Jan 10, 2017 at 15:26
  • $\begingroup$ Requiring soft-outputs requires a significant higher complexity and it is not natively available in Matlab. Yes, I think you would need to write this piece on your own, using the BCJR or similar approaches. $\endgroup$ Jan 10, 2017 at 18:04
  • $\begingroup$ Thanks. I assume this function considers noise samples are white. If not while, then I need to change the effective channel input. I guess it will be convolution between original channel and whitening filter coefficients, right? $\endgroup$
    – Noor
    Jan 10, 2017 at 18:11
  • $\begingroup$ Well, I believe it assumes white noise. But, since it does not produce soft-output, it also has no parameter for the noise variance. In the documentation I could not find information about the assumed noise model. Right, if the noise were correlated, you'd need to send the signal through a whitening filter before (which is most likely an IIR filter), which increases the effective channel length and might even reduce performance. You would need to try this out. It's a non-trivial problem. $\endgroup$ Jan 10, 2017 at 20:46
  • $\begingroup$ Thanks a lot for your time. Do you recommend some good resources on these DSP and coding theory concepts? $\endgroup$
    – Noor
    Jan 10, 2017 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.