You have some predefined commands to change the fonts of some elements:
\namefont
\titlefont
\addressfont
\sectionfont
\subsectionfont
\hintfont
\quotefont
Redefining those commands you can change the font attributes for the associated elements. Here's a little example in which I changed the font associated to the first four commmands to use Zapf Chancery:
\documentclass[11pt,a4paper,sans]{moderncv}
\moderncvstyle{classic}
\moderncvcolor{blue}
\usepackage[scale=0.75]{geometry}
\usepackage[T1]{fontenc}
\firstname{John}
\familyname{Doe}
\title{CV}
\address{Baker Street}{Southampton}
\mobile{+1~(234)~567~890}
\phone{+2~(345)~678~901}
\fax{+3~(456)~789~012}
\email{john@doe.org}
\homepage{www.johndoe.com}
\renewcommand*\namefont{\fontfamily{pzc}\fontsize{40}{48}\selectfont}
\renewcommand*\titlefont{\fontfamily{pzc}\fontsize{20}{24}\selectfont}
\renewcommand*\addressfont{\fontfamily{pzc}\selectfont}
\renewcommand*\sectionfont{\fontfamily{pzc}\fontsize{20}{24}\selectfont}
\begin{document}
\maketitle
\section{Test Section}
\cventry{2012}{Title}{Institute}{City}{}{Description}
\cventry{2012}{Title}{Institute}{City}{}{Description}
\section{Test Section}
\cventry{2012}{Title}{Institute}{City}{}{Description}
\cventry{2012}{Title}{Institute}{City}{}{Description}
\end{document}

If you want to change the font for some particular argument(s) of, for example, the \cventry command you can define your own command to include the necessary redefinition(s); here's an example in which I defined a \Mycventry command using Zapf Chancery for the last argument of \cventry:
\documentclass[11pt,a4paper,sans]{moderncv}
\moderncvstyle{classic}
\moderncvcolor{blue}
\usepackage[scale=0.75]{geometry}
\usepackage[T1]{fontenc}
\usepackage{lipsum}
\firstname{John}
\familyname{Doe}
\title{CV}
\address{Baker Street}{Southampton}
\mobile{+1~(234)~567~890}
\phone{+2~(345)~678~901}
\fax{+3~(456)~789~012}
\email{john@doe.org}
\homepage{www.johndoe.com}
\newcommand\Mycventry[6]{%
\cventry{#1}{#2}{#3}{#4}{#5}{\fontfamily{pzc}\selectfont#6}}
\begin{document}
\maketitle
\section{Test Section}
\Mycventry{2012}{Title}{Institute}{City}{}{\fontfamily{pzc}\selectfont \lipsum[2]}
\cventry{2012}{Title}{Institute}{City}{}{\lipsum[2]}
\end{document}
And the output showing the comparison between \cventrya and \Mycventry:

\cventry". Please see my updated answer. – Gonzalo Medina Mar 28 '12 at 18:03