Instead of using the obsolete subfigure package you should use subfig or subcaption; here's your code using subfig:
\documentclass{article}
\usepackage{amsmath}
\usepackage{subfig}
\usepackage[demo]{graphicx}
\usepackage{caption}
\renewcommand{\thesection}{\arabic{section}.}
\renewcommand{\thesubsection}{\thesection\arabic{subsection}.}
\renewcommand{\thefigure}{\arabic{section}.\arabic{figure}}
\makeatletter
\renewcommand{\section}{%
\@startsection
{section}{1}{0mm}
{\baselineskip}
{\baselineskip}
{\fontsize{14}{14}\centering\bfseries\MakeUppercase}%
}
%subsection
\renewcommand{\subsection}{%
\@startsection
{subsection}{1}{0mm}
{\baselineskip}
{\baselineskip}
{\fontsize{14}{14}\centering\bfseries\MakeUppercase}%
}
\makeatother
\begin{document}
\section{a}
\begin{figure}[h]
\centering
\subfloat[]{\label{fig:11}%
\includegraphics[width=60mm]{1.png}}\ %
\subfloat[]{\label{fig:12}%
\includegraphics[width=60mm]{2.png}}
\caption{11 and 12}
\label{fig:1}
\end{figure}
\ref{fig:11} or \ref{fig:1}\subref{fig:11}
\end{document}

As a side note, your redefinition for \thefigure won't reset the counter when a new section begins; if you want to reset the counter you could use
\@addtoreset{figure}{section}
inside \makeatletter, \makeatother, or
\numberwithin{figure}{section}
from the amsmath package, or
\counterwithin{figure}{section}
from the chngcntr package. Also note that, inside floating environments, \label must always appear after \caption (in your code \label is before \caption).
If, for some reason you need to stick to the subfigure package, then you can redefine \thesubfigure, \p@subfigure, and \@thesubfigure:
\documentclass{article}
\usepackage{amsmath}
\usepackage{subfigure}
\usepackage[demo]{graphicx}
\usepackage{caption}
\renewcommand{\thesection}{\arabic{section}.}
\renewcommand{\thesubsection}{\thesection\arabic{subsection}.}
\renewcommand{\thefigure}{\arabic{section}.\arabic{figure}}
\renewcommand{\thesubfigure}{\thefigure(\alph{subfigure})}
\makeatletter
\renewcommand{\p@subfigure}{}
\renewcommand{\@thesubfigure}{(\alph{subfigure})\hskip\subfiglabelskip}
\makeatother
\makeatletter
\renewcommand{\section}{%
\@startsection
{section}{1}{0mm}
{\baselineskip}
{\baselineskip}
{\fontsize{14}{14}\centering\bfseries\MakeUppercase}%
}
%subsection
\renewcommand{\subsection}{%
\@startsection
{subsection}{1}{0mm}
{\baselineskip}
{\baselineskip}
{\fontsize{14}{14}\centering\bfseries\MakeUppercase}%
}
\makeatother
\begin{document}
\section{a}
\begin{figure}[h]
\centering
\subfigure[]{\label{fig:11}%
\includegraphics[width=60mm]{1.png}}\ %
\subfigure[]{\label{fig:12}%
\includegraphics[width=60mm]{2.png}}
\caption{11 and 12}
\label{fig:1}
\end{figure}
\ref{fig:11}
\end{document}

\subfigtopskipis not defined insubfignor insubcaption; but do you really want to add1cmbefore the subfigures? – Gonzalo Medina Jun 05 '12 at 21:57subfigure; in my updated answer I've provided the necessary code withsubfigure. However, for future documents, I recommend switching to eithersubfigorsubcaption. – Gonzalo Medina Jun 05 '12 at 22:14subfigureis incompatible with really important packages such ashyperrefandcaption. – Gonzalo Medina Jun 07 '12 at 03:05