Here's a patch that works for me:
$ cat fancyvrb.sty.patch
992c992,993
< \ifx\@tempa\@tempb
---
> \def\FV@FirstNumber{#1}%
> \ifx\@tempa\@tempb%
1128,1129c1129,1139
< \FV@TheVerbatim\FV@ProcessLine{##1}}}%
< \gdef\FV@TheVerbatim{}%
---
> \FV@TheVerbatim\advance\c@FancyVerbLine\@ne\FV@ProcessLine{##1}}}%
> \gdef\FV@TheVerbatim{%
> \edef\@tempa{\FV@FirstNumber}%
> \def\@tempb{auto}%
> \ifx\@tempa\@tempb%
> \c@FancyVerbLine\FV@CodeLineNo%
> \else%
> \c@FancyVerbLine\FV@FirstNumber%
> \advance\c@FancyVerbLine\m@ne%
> \fi%
> }%

If you apply it locally, for example, by copy/pasting the following into a unix shell prompt:
cat > fancyvrb.sty.patch
992c992,993
< \ifx\@tempa\@tempb
---
> \def\FV@FirstNumber{#1}%
> \ifx\@tempa\@tempb%
1128,1129c1129,1139
< \FV@TheVerbatim\FV@ProcessLine{##1}}}%
< \gdef\FV@TheVerbatim{}%
---
> \FV@TheVerbatim\advance\c@FancyVerbLine\@ne\FV@ProcessLine{##1}}}%
> \gdef\FV@TheVerbatim{%
> \edef\@tempa{\FV@FirstNumber}%
> \def\@tempb{auto}%
> \ifx\@tempa\@tempb%
> \c@FancyVerbLine\FV@CodeLineNo%
> \else%
> \c@FancyVerbLine\FV@FirstNumber%
> \advance\c@FancyVerbLine\m@ne%
> \fi%
> }%
C-c C-c
patch -o fancyvrb.sty $(kpsewhich fancyvrb.sty) fancyvrb.sty.patch
I've also emailed this to the package maintainer (Herbert Voß), maybe they could tell me how to proceed towards a proper patch/fix.