Note: I rewrote this answer significantly after becoming aware of Ulrike Fisher's comment to @knut's answer.
There are two steps that need to be taken. The first, straightforward step consists of redefining the \thesubsection macro. The second, less obvious step involves a redefinition of the LaTeX internal macro \@seccntformat, as is explained in the book The LaTeX Companion, 2nd ed. The following MWE applies both steps:

\documentclass{article}
\renewcommand{\thesubsection}{\arabic{subsection}}
\makeatletter
\def\@seccntformat#1{\@ifundefined{#1@cntformat}%
{\csname the#1\endcsname\quad}% default
{\csname #1@cntformat\endcsname}}% enable individual control
\newcommand\section@cntformat{} % section level
\makeatother
\begin{document}
\section{First Section}
\subsection{First subsection}
\subsection{Second subsection}
\subsubsection{First subsubsection}
\subsection{Third subsection}
\section{Second Section}
\subsection{A new subsection}
\end{document}
Addendum to address @Adam's follow-up question: To (a) add dots after the subsection (and subsubsection) numbers in the sectioning headers and (b) replace the implicit \quad spacing directive with \space, you would need to provide the following two directives in the preamble, immediately before \makeatother:
\newcommand\subsection@cntformat{\thesubsection.\space}
\newcommand\subsubsection@cntformat{\thesubsubsection.\space}
A revised MWE:

\documentclass{article}
\renewcommand{\thesubsection}{\arabic{subsection}}
\makeatletter
\def\@seccntformat#1{\@ifundefined{#1@cntformat}%
{\csname the#1\endcsname\quad}% default
{\csname #1@cntformat\endcsname}}% enable individual control
\newcommand\section@cntformat{} % section level
\newcommand\subsection@cntformat{\thesubsection.\space} % subsection level
\newcommand\subsubsection@cntformat{\thesubsubsection.\space} % subsubsection level
\makeatother
\begin{document}
\section{First Section}
\subsection{First subsection}
\subsection{Second subsection}
\subsubsection{First subsubsection}
\subsection{Third subsection}
\section{Second Section}
\subsection{A new subsection}
\end{document}
\section*? Should the numbering start again at 1 or should it continue? – knut Dec 05 '11 at 21:39