We start off with the hint that @WhatsUp gave. Note that for all $n$ we have
$$\frac{1}{n}\left[f\left(x-\frac{1}{2n}\right)+f\left(x+\frac{1}{2n}\right)\right]=\int_{x-\frac{1}{n}}^{x}f(t)dt+\int_{x}^{x+\frac{1}{n}}f(t)dt=\int_{x-\frac{1}{n}}^{x+\frac{1}{n}}f(t)dt=\frac{2}{n}f(x)$$
$$\Rightarrow f(x)=\frac{f\left(x-\frac{1}{2n}\right)+f\left(x+\frac{1}{2n}\right)}{2}$$
Now, fix $f(0)=a$ and $f(1)=b$. The details are slightly tedious, but we are able to show by induction that the above equation as well as the knowledge of $f(0)$ and $f(1)$ imply
$$f(x)=(b-a)x+a$$
for all
$$x\in S=\left\{\frac{k}{2^r}:k\in\mathbb{Z}\text{ and }r\in\mathbb{N}\right\}$$
(see below for induction proof). Going forward, define $g(x)=f(x)-(b-a)x-a$. Thus, $g(x)$ is $0$ on the set $S$ and $g(x)$ satisfies
$$g(x)=f(x)-(b-a)x-a=\frac{n}{2}\int_{x-\frac{1}{n}}^{x+\frac{1}{n}}f(t)dt-(b-a)x-a$$
$$=\frac{n}{2}\int_{x-\frac{1}{n}}^{x+\frac{1}{n}}[g(t)+(b-a)t+a]dt-(b-a)x-a=\frac{n}{2}\int_{x-\frac{1}{n}}^{x+\frac{1}{n}}g(t)dt$$
the same integral relationship as $f(x)$. Now, let $y$ be any real not already in $S$ and let $y_m$ be a sequence of rational numbers of the form $\frac{k}{2^r}$ (for integral $k$ and $r$) that approaches $y$. Further, define $z_m=y-y_m$ (note that $z_m$ approaches $0$). Then
$$g(y)=\frac{1}{2}\int_{y-1}^{y+1}g(t)dt$$
$$=\frac{1}{2}\int_{y_m-1}^{y_m+1}g(t)dt+\frac{1}{2}\int_{y-1}^{y-1-z_m}g(t)dt+\frac{1}{2}\int_{y+1-z_m}^{y+1}g(t)dt$$
$$=g(y_m)+\frac{1}{2}\int_{y-1}^{y-1-z_m}g(t)dt+\frac{1}{2}\int_{y+1-z_m}^{y+1}g(t)dt$$
$$=\frac{1}{2}\int_{y-1}^{y-1-z_m}g(t)dt+\frac{1}{2}\int_{y+1-z_m}^{y+1}g(t)dt$$
But by the first assumption, we know $g(t)$ is Riemann integrable. This implies that for $m$ such that $|z_m|\leq 1$ we have
$$\left|\frac{1}{2}\int_{y-1}^{y-1-z_m}g(t)dt+\frac{1}{2}\int_{y+1-z_m}^{y+1}g(t)dt\right|\leq z_m \sup_{t\in (y-4,y+4)}g(t)$$
Again, since $g(x)$ is Riemann integrable, the supremum of $g(x)$ over the interval $(y-4,y+4)$ must exist (see here for a good description regarding this). This implies
$$0\leq |g(y)|\leq z_m \sup_{t\in (y-4,y+4)}g(t)$$
Since this is true for all but a finite number of $m$ and $z_m$ approaches $0$, we conclude
$$0\leq |g(y)|\leq \lim_{m\to\infty} z_m \sup_{t\in (y-4,y+4)}g(t)=0$$
Thus, $g(y)=0$ and we are done.
Induction proof: We will proceed with a double induction on the tuples $(k,r)$ for $k\in\mathbb{Z}$ and $r\in\mathbb{N}$.
First, note that at $n=1$ we have
$$f\left(\frac{1}{2}\right)=\frac{f(0)+f(1)}{2}=\frac{b+a}{2}=\frac{b-a}{2}+a$$
as desired. Thus, the proposition is true for $(0,1),(1,1),(2,1)$. Assume it is true for $(k,1)$ such that $k\in \{0,1,...,m\}$. Then
$$f\left(\frac{m}{2}\right)=\frac{f\left(\frac{m+1}{2}\right)+f\left(\frac{m-1}{2}\right)}{2}$$
Then solving for our unknown gives us
$$f\left(\frac{m+1}{2}\right)=2f\left(\frac{m}{2}\right)-f\left(\frac{m-1}{2}\right)$$
$$=(b-a)m+2a-(b-a)\frac{m-1}{2}-a=(b-a)\left(\frac{m+1}{2}\right)+a$$
as desired. A similar construction works for negative $k$. Now, assume the proposition is true for all $(k,r)$ where $k\in\mathbb{Z}$ and $r\in\{1,2,...,m\}$. If $k$ is even, then the proposition is proved since for $k=2s$ we have
$$\frac{k}{2^{m+1}}=\frac{2s}{2^{m+1}}=\frac{s}{2^m}$$
If not, then for $n=2^m$ we have
$$f\left(\frac{k}{2^{m+1}}\right)=\frac{f\left(\frac{k}{2^{m+1}}+\frac{1}{2^{m+1}}\right)+f\left(\frac{k}{2^{m+1}}-\frac{1}{2^{m+1}}\right)}{2}$$
$$=\frac{f\left(\frac{k+1}{2}\cdot\frac{1}{2^{m}}\right)+f\left(\frac{k-1}{2}\cdot\frac{1}{2^{m}}\right)}{2}$$
Note that both $\frac{k\pm 1}{2}$ are integral since $k$ is odd. Then by our inductive assumption we have
$$=\frac{(b-a)}{2}\left[\frac{k+1}{2}\cdot \frac{1}{2^m}+\frac{k-1}{2}\cdot \frac{1}{2^m}\right]+\frac{1}{2}(a+a)=(b-a)\frac{k}{2^{m+1}}+a$$
as desired.