7

I'm going over some of the pre-requisite math regarding Automata theory, and finite representations.

I read the following:

  • If ∑ is a finite alphabet the set of all strings over the alphabet (∑*) is countably infinite.

  • The set of all possible languages over an alphabet ∑ is uncountably infinite.

How can the set of languages possible from ∑ be uncountably infinite, yet the possible application of that alphabet to a language be countably infinite?

Can I ask those replying to not use too much complex notation, as I'm not a mathematics wizz.

Raphael
  • 72,336
  • 29
  • 179
  • 389
Andrew S
  • 205
  • 2
  • 5
  • 2
    Do you understand the difference between countable infinite and uncountable infinite, or is there only one kind of infinite in your mind? Do you have a vague idea of how this distinction came to be (cf. Cantor)? –  Mar 04 '14 at 23:43
  • Thanks. I kind of get the idea of uncountably infinite. It essentially is a set that cannot be indexed, like a set of real numbers, because say the possible number of incremental steps between say 1.0 and 2.0 is immeasurably infinite. – Andrew S Mar 04 '14 at 23:49
  • 1
    "Immeasurably infinite" is not a technical term AFAIK (if it is, disregard the following). If by that you mean that there's a (countably or not) infinite number of steps between 1 and 2, then you've fallen into the trap of underestimating infinity. The real number interval [1;2] is indeed uncountable, but the rationals in the same interval are countable, despite there being a (countably) infinite number of them and no discrete steps! –  Mar 04 '14 at 23:56

2 Answers2

5

Here is a simpler situation highlighting the difference. The set of finite binary strings is countable. The set of infinite binary strings is uncountable.

Another example: the set of numbers with finite decimal expansion is countable. The set of numbers with infinite decimal expansion is uncountable.

The reason that the number of languages is uncountable is that you have infinitely many choices: for each word, you can decide whether it's in the language or not. That's why it's like that infinite binary string considered above.

Yuval Filmus
  • 276,994
  • 27
  • 311
  • 503
  • The first two arguments are not convincing, an infinite set of binary strings is countable just infinitely, 0, 1, 10, 11, .....infinity (correctly me if I'm wrong,which I probably am). However, I like your last example. – Andrew S Mar 05 '14 at 01:42
  • 2
    The set of all finite strings is countable - that's 0, 1, 10, 11, ... The set of all infinite strings is uncountable. You can't even write any of them, since they're all infinite. Examples can be $0^\omega$ (infinitely many zeroes), $(01)^{\omega}$ (a repeating pattern), $1010^210^310^410^5\dots$ (a non-repeating pattern), and many, many more. Uncountably more. More than you can describe. – Yuval Filmus Mar 05 '14 at 01:52
  • 1
    @AndrewS, you are considering finite binary strings. Check out Cantor's diagonal argument to see the crucial difference. – vonbrand Mar 05 '14 at 01:53
  • Thanks so much folks I'll now sleep tonight (well a little bit). – Andrew S Mar 05 '14 at 02:06
  • I don't think this answer provides a valuable intuition. 1) There are non-trivial countable sets of infinite strings, e.g. represented by Turing machines or Büchi automata. 2) The rational numbers contain such that have infinite decimal expansions, yet the set is countable. 3) The property of "having infinitely many choices" is too vague to be useful. – Raphael Mar 05 '14 at 07:14
  • Regarding 3), In fact $\prod_{i \in \mathbb{N}} {0,1}$ is uncountable, and this is intuitively why all kinds of sets are uncountable. – Yuval Filmus Mar 05 '14 at 12:04
4

Questions of the form "how can" are always hard to answer as they ask for intuition. The intuition here is: uncountable means "too many" in way. Why are there "much more" objects of one kind than of the other kind? Well, they just are.

Make sure you understand the facts.

  1. Given a finite set $\Sigma$, construct the bijection to $\mathbb{N}$ for the set of all (finite) strings over $\Sigma^*$.

    Hint: $\Sigma^n$ is finite, so combining a generalised numbering of $\Sigma^n$ with $n$ should work.\

  2. Show that the set of all tuples of naturals, i.e. $\mathbb{N}^+ = \bigcup_{i \geq 1} \mathbb{N}^i$, is countable.

    Hint: Construct the bijection for $\mathbb{N}^i$ first (cf. Cantor) and combine those, again using $i$.

  3. Show that the power set of $\mathbb{N}$, i.e. $2^{\mathbb{N}}$, is uncountable.

    Hint: Use diagonalisation.

If you have done this, you have all the tools to show for instance that

  • all algorithmic (or, more generally, finitely represented) concepts are countable but
  • the set of all functions over naturals or reals or ... is uncountable.
Community
  • 1
Raphael
  • 72,336
  • 29
  • 179
  • 389