You probably don't want the rule before the first section, nor when a section starts at the top of a page. The following code accomplishes this.
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{titlesec}
\usepackage{lipsum}
\titleformat{\section}
{\sectionrule\Large\bfseries}
{\thesection}
{1em}
{}
% this command is executed at each \section command
\newcommand{\sectionrule}{%
% no rule before the first section
\ifnum\value{section}=0
\else
% otherwise, ensure being between paragraphs
\par
% add some vertical space
\addvspace{\bigskipamount}%
% the rule realized as leaders, so it disappears at a page break
% see also http://tex.stackexchange.com/a/61643/4427
\leaders\vrule width \textwidth\vskip0.4pt
% some other vertical space
\bigskip
\fi
}
\begin{document}
\section{First}
\lipsum[1-3]
\section{Second}
\lipsum[2-6]\lipsum[2]\lipsum[2]
\section{Third}
\lipsum
\end{document}

Alternate solution, with the rule at the top of the page in case of a page break.
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{titlesec}
\usepackage{lipsum}
\titleformat{\section}
{\sectionrule\Large\bfseries}
{\thesection}
{1em}
{}
\newcommand{\sectionrule}{%
\par
\addvspace{\bigskipamount}%
\hrule
\nopagebreak
\bigskip
}
\begin{document}
\section{First}
\lipsum[1-3]
\section{Second}
\lipsum[2-6]\lipsum[2]\lipsum[2]
\section{Third}
\lipsum
\end{document}
