Adding the option:
[captions=nooneline]
fixes the problem.
I removed captions=, when I revised the answer (added more caption option), and it stopped working again. Confusing. If you remove \footnotesize, it also works as expected. This is not a bug, as @esdd explain in the comment below. With default captions=oneline, \setcapwidth only effects captions that is more than one line, i.e: wider than the text width. So when you reduce font size as you did, the caption fits within the text width again, and becomes a one line caption, even if it is wider than the value you gave \setcapwidth. It will not be changed by \setcapwidth. Good to know!
In stead of [captions=nooneline], you may use a centred, \minipage-environment of the width you prefer (here 8cm), and centre the table within that environment:
Example 1 – centred minipage

\documentclass[captions=nooneline, captions=tableabove]{scrbook}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{lmodern}
\usepackage{booktabs, relsize}
\setkomafont{caption}{\sffamily\footnotesize} % This is the line in question
\begin{document}
\begin{table}[tbp]
\centering
\footnotesize
\begin{minipage}{8cm}
\centering
\caption{This is a rather long caption for a table that is comparatively narrow --- but it's a test}
\label{table:setcapwidth_table}
\begin{tabular}{@{} l r r @{}}
\toprule
Row 1 & Row 2 & Row 3 \\
\midrule
2011& 10,000 & 10,000 \\
2012 & 10,000 & 10,000 \\
2013 & 10,000 & 10,000 \\
\bottomrule
\end{tabular}
\end{minipage}
\end{table}
\end{document}
Example 2 – threeparttable
Also, you may enclose the tabular-environment in a threeparttable if you prefer to match the width of the caption with the width of the tabular. As an added benefit, you may have nice, integrated table notes.

\documentclass[captions=nooneline, captions=tableabove]{scrbook}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{lmodern}
\usepackage{booktabs, threeparttable}
\setkomafont{caption}{\sffamily\footnotesize} % This is the line in question
\begin{document}
\begin{table}[tbp]
\footnotesize
\centering
\begin{threeparttable}
\caption{This is a rather long caption for a table that is comparatively narrow --- but it's a test}
\label{table:setcapwidth_table}
\begin{tabular}{@{} l r r @{}}
\toprule
Row 1 & Row 2 & Row 3 \\
\midrule
2011& 10,000 & 10,000 \\
2012 & 10,000 & 10,000 \\
2013 & 10,000 & 10,000 \\
\bottomrule
\end{tabular}
\end{threeparttable}
\end{table}
\end{document}