4

From "Misconception 1" from Søren S. Pedersen's blog, and as many have seen before, a major misconception in Big-O (and others) notation is to say a function is "equal" to Big-O of some other function:

$f(n) = O(g(n))$

where what we actually mean is this:

$f(n) \in O(g(n))$

What is the origin of this misconception? And why is this misconception perpetuated, even today?

Raphael
  • 72,336
  • 29
  • 179
  • 389
Ryan Dougherty
  • 1,013
  • 8
  • 18
  • 3
    It's not a misconception, but rather standard notation that unfortunately overloads the equality sign with a relation which isn't an equivalence relation. – Yuval Filmus Sep 13 '14 at 22:15
  • @Yuval: I think that the misconception here is one that, let's say, a student would make when first looking at it. – mdxn Sep 14 '14 at 03:37
  • @mdx : That may not be correct to call it a misconception due to lack of knowledge. If + operator is overloaded to concatenate strings, is it a misconception or overloading to keep idea general? – vish213 Sep 14 '14 at 03:40
  • It's calculated sloppiness that has become an unfortunate standard (maybe because of the prevalence of CLRS?) to the point of actually being used wrong (imho because it's all but impossible to teach precision this way; if anything, $\leq O(\dots)$ should be used). Don't use it if you are uncomfortable with it (I know I don't), but you have to be aware of what most people mean by it. – Raphael Sep 15 '14 at 10:28
  • 1
    See also this elaborate discussion on math.SE which proposes some "fixes". The issue is not as easily dismissed as some may think. – Raphael Sep 15 '14 at 10:31

2 Answers2

8

$O$ is not only used in simple statements like $f(n)=O(g(n))$. It is also used to give error terms as in $f(n) = 14\, n\log n + O(n)$. The interpretation is still the same as @usul has described in his answer: There is a function $h \in O(n)$ so that $f(n) = 14\, n\log n + h(n)$.

Here, writing $f(n) \in 14\, n\log n + O(n)$ would emphasize the fact that there is a set involved in the equation, while the important point is that the leading term(s) are exactly as stated, while the $O$-term simply bounds the error. So in this context using $=$ is a sensible choice and I'd assume that it has been carried over to the case with zero leading terms in order to keep notation consistent.

FrankW
  • 6,589
  • 4
  • 26
  • 42
  • This is pretty much the only use case I find acceptable, as long as the error term is in $o$ of the explicit terms. Then, it bears resemblance to $a = b \pm \varepsilon$. (Maybe one should actually use $\pm$ in such cases.) – Raphael Sep 15 '14 at 10:29
2

As Yuval comments, "misconception" isn't the right term for the notation. It's well-defined notation, which I agree is somewhat bad and unfortunate, but it's valid. It might make sense once we say that, whenever you see a $O(\cdot)$ in a statement, you are being told that there exists a choice from the set $O(\cdot)$ that makes the statement true. So you can read $f(n) = O(g(n))$ as "there exists $h(n) \in O(g(n))$ such that $f(n) = h(n)$.

This may seem silly for this simple example (we could just write $f(n) \in O(g(n))$), but I think it can come in handy with more complex statements. I am thinking of proving big-O containment over a sequence of steps: \begin{align} f(n) &= ... \\ &\leq ... \\ &= O(...) \\ &= O(g(n)). \end{align} Maybe this seems nicer or prettier than trying to use $\in$ and $\subseteq$ etc. on each line.

I would guess that this, along with the fact that it's slightly quicker to write $=$ than $\in$, and easier in e.g. plain text environments, accounts for the popularity of the notation. But a historical perspective would be interesting.

usul
  • 4,129
  • 21
  • 30
  • "Maybe this seems nicer or prettier than trying to use..." -- true. So why not stop at $\leq \dots$ and continue with "so $f(n) \in O(\dots)$ or give the witness from $O(\dots)$ a name? – Raphael Sep 15 '14 at 10:40