The book "Statistical Learning Theory" by Vladimir Vapnik has a part which I cannot understand properly.
"According to the classical law of large numbers, the frequency of any event converges to the probability of this event with an increasing number of observations. However, the classical law of large numbers is not sufficient to assert that for a given set of event the sequence of probabilities of events with the smallest frequency converges to the smallest possible value for this set."
The first line is pretty self-explanatory. The next line describes a limitation of the classical law of large numbers. What I understood is that if there are many events, and some event has the smallest frequency for this set, the law does not assert that this event will also have the smallest probability in the set (with increasing observations). But I don't see why it can't be asserted. Also, why is there a mention of sequence of probabilities? The first line creates a distinction between frequency and probability. How can there be a sequence of probabilities related to an event? Can anyone care to provide further insight on this?
