Confusion over the definition of "source code" in information theory

Question

This is from Thomas and Cover's textbook, 2nd.

Def. A source code $C$ for a random variable $X$ is a mapping from $\mathcal{X}$, the range of $X$, to $\mathcal{D}^\star$, the set of finite length strings of symbols from a $D$-ary alphabet.

Ok. Then the example immediately following is:

$C(red) = 00$, $C(blue) = 11$ is a source code for $\mathcal{X} = \{red, blue\}$ with alphabet $\mathcal{D} = \{0, 1\}.$

What is an "alphabet" and a "symbol" in this definition?
Since the range of random variables are subsets of $\mathbb{R}$, then how can $C$ map from blue and red when these are not element of $\mathbb{R}$?

Not an expert on this subject, but I think I’d define a “$D$-ary alphabet” to be just a set with $D$ elements. In this context, once an alphabet has been introduced, the elements of the alphabet are called “symbols”. — littleO, Oct 11 '22 at 07:16

score 0 · Answer 1 · answered Oct 11 '22 at 01:34

0

What is an "alphabet" and a "symbol" in this definition?

In general, an alphabet is the repertoire (the set) of possible values.

In this case, the source can take two values $\{ red, blue\}$ , this would be $\mathcal{X}$, the source alphabet.

And the code is binary, then the symbols are $\{ 0, 1 \}$, and this is the code alphabet ($\mathcal{D}$).

Since the range of random variables are subsets of $\mathbb{R}$...

What? No. The range of a random variable can be anything.

answered Oct 11 '22 at 01:34

leonbloy

63,430

The range of random variable must be $\mathbb{R}$ no? https://math.stackexchange.com/questions/240673/what-exactly-is-a-random-variable. https://math.stackexchange.com/questions/3777899/so-what-really-is-a-random-variable – Sin Nombre Oct 11 '22 at 01:49
1

The range need not be $\mathbb R$. https://en.wikipedia.org/wiki/Random_variable – GEdgar Oct 11 '22 at 07:03
No, the range must not be real. See the first two examples in the most upvoted answer you link. See also https://math.stackexchange.com/a/3777902/312
It's true than in most applications we deal with real-valued (perhaps multidimensional) random variables (and some concepts like expected value make sense only for them) https://math.stackexchange.com/questions/4104788/definition-of-expectation-for-non-real-valued-random-variables
– leonbloy Oct 11 '22 at 11:17
@leonbloy Thanks for the references. But non-real-valued rvs seem to be obscure, no? You could define it but I don't see any real applications arising from them. There also doesn't seem to be a textbook that deals with these rvs. – Sin Nombre Oct 11 '22 at 14:02
"I don't see any real applications arising from them" Well, take information theory :-) Contrary to other measures like expectation, variance, etc, Shannon entropy does not imply or require a real-valued variable. – leonbloy Oct 11 '22 at 14:12
To the contrary, all kinds of signals are complex, QAM, QPSK, etc, and the main focus of information theory is digital communications. The alphabet may be ${1,-1,i,-i}$ or other complex sets. – kodlu Oct 11 '22 at 18:36

Confusion over the definition of "source code" in information theory

1 Answers1

Linked