1
$\begingroup$

In Shannon's paper "A Mathematical Theory of Communication", in Theorem 8 he states:

Theorem 8: Let the system of constraints considered as channel have a capacity $C = \log W$. If we assign $$p^{(s)}_{ij} = \frac{B_j}{B_i}W^{\ell^{(s)}_{ij}}$$ where $l^{(s)}_{ij}$ is the duration of the $s^{th}$ symbol leading from state i to state j and $B_i$ satisfy $$B_i = \Sigma_{s,j}B_jW^{\ell^{(s)}_{ij}}$$ then H is maximized and equal to C.

My question is, what exactly is $B_i$ and $B_j$? It's not defined anywhere else. In a different section he mentions "a sequence $B_i$ of symbols from the source", but that doesn't make much sense here.

$\endgroup$
1
$\begingroup$

Charles E.Radke, "Necessary and sufficient conditions on conditional probabilities to maximize entropy", Information and Control 9, 279-284 (1966), https://doi.org/10.1016/S0019-9958(66)90165-3 gives a proof of Shannon's Theorem 8:

A necessary and sufficient condition such that $\eta_c$ is maximized and equal to $1$ is that the set of symbol lengths ${l_{ij}}^v$ must satisfy: $${p_{ij}}^v = (\beta_j/\beta_i)\omega_r^{-{l_{ij}}^v},\tag{3}$$ where $\beta_i$ and $\beta_j$ are respectively the $(k, i)$th and $(k, j)$th cofactor of the matrix $A$ for any $k$.

The matrix $A$ is defined earlier as:

$$A \triangleq \left[\textstyle\sum\omega^{-{l_{ij}}^v} - \delta_{ij}\right];$$

which is something you can recognize in Shannon's Theorem 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.