Don't forget that we're working in a Hilbert space - $L^2(\mathbb R)$, in this case. The operators $\hat A$ can only act on elements in $L^2(\mathbb R)$, but their domains are typically further restricted to those $\psi \in L^2(\mathbb R)$ such that $\Vert A \psi \Vert^2 < \infty$ and, for more general Hilbert spaces, further still by boundary conditions.
Mathematical Interlude
As the notes mention, this is a rather restrictive requirement. In many cases, it turns out to be extremely useful to extend this notion slightly by considering distributions on $L^2(\mathbb R)$. A distribution$^\dagger$ is an object which eats an element of $L^2(\mathbb R)$ and spits out a complex number, and can typically be written as
$$D_\phi[\psi] = \int_{-\infty}^\infty \phi^*(x) \psi(x) dx$$
for some function $\phi$, which we might call the kernel of the distribution (but usually we are sloppy and just call it "the distribution $\phi$"). In this expression, $\psi$ must belong to $L^2(\mathbb R)$, but the restrictions on $\phi$ are looser. Of course, $\phi$ could be an element of $L^2(\mathbb R)$, in which case this expression reduces to the inner product $\langle \phi,\psi\rangle$, but this is not always the case; in particular, $\phi$ generally need not be normalizable. In particular, delta functions$^{\dagger\dagger}$ $\delta(x-x_0)$ and plane waves $e^{ikx}$ are common distribution kernels.
Having done this, we can define the action of a self-adjoint operator $\hat A$ on $\phi$ as follows:
$$D_{\hat A\phi}[\psi] := D_\phi[\hat A\psi]$$
$$\iff \int_{-\infty}^\infty (\hat A\phi)^*(x) \psi(x) dx = \int_{-\infty}^\infty \phi^*(x) (\hat A\psi)(x) dx$$
Operators such as the position and momentum observables do not actually have any eigenvectors in $L^2(\mathbb R)$, but if we extend them in this way then it turns out that they do have "generalized eigenvectors" which are distributions (the delta functions and plane waves, respectively).
This is, of course, too much math for a first pass through quantum mechanics. In practice, this can be distilled down to the following:
- Observables with continuous spectra do not have eigenvectors in $L^2(\mathbb R)$ (that is, they do not have normalizable eigenvectors), but if we allow them to act on distributions (which do not have the normalizability requirement), then they have generalized eigenvectors.
- In order for this procedure to be well-defined, we need certain regularity requirements on the distributions. They need not be normalizable, but we must have that $D_{\hat A\phi}[\psi] = D_\phi[\hat A\psi] = \lambda D_\phi[\psi]$ is finite for all $\psi$ in the domain of $\hat A$.
- It turns out that the proper choices for the regularity requirements of the Hamiltonian $\hat H = -\frac{d^2}{dx^2} + V(x)$ are boundedness of the distribution and its first derivative, as per the notes.
$^\dagger$ In the interest of completeness, we usually consider tempered distributions, which only act on a very well-behaved subset of $L^2(\mathbb R)$. See the second to last paragraph in the introduction of this wikipedia article. However, I feel that this detail is too far removed from the original question to be necessary.
$^{\dagger\dagger}$As noted by TBissinger, the delta function is not actually a function. Instead, the delta distribution is defined via
$$\delta_{x_0}[\psi] := \psi(x_0)$$
That is, it is the distribution which simply evaluates the function at a point. In order to make this look like other distributions which have kernels, we define the "delta function" and write
$$\delta_{x_0}[\psi] := \psi(x_0) \simeq \int_{-\infty}^\infty \delta(x-x_0) \psi(x) dx$$
and simply note that $\delta(x-x_0)$ is a purely formal symbol which should not be thought of as a function in the usual sense.
