As Seamus mentions in a comment, you can do this by adding a LaTeX command for changing font size, e.g. \small just before the bibliography. To add (La)TeX code in a LyX document, one has to write the code in a TeX code box, also known as an ERT (Evil Red Text), which is inserted by Insert --> TeX code, or the keyboard shortcut Ctrl + L.
However (see comment from Stephen), note that as \small and friends are just switches, all text that follows will also have the new font size. Hence, any text that is after the bibliography will also be \small. To make the change local to the bibliography, write a { before \small in the ERT, and add a second ERT just after the bibliography where you write a closing brace, }.
This of course only necessary if you actually have anything after the bibliography.
See screenshot below.
Reference

\smallor similar before the\bibiliographycommand. But I don't use LyX, it might have made things unnecessarily complicated. – Seamus Jan 14 '13 at 13:00