The problem seems the unexpanded name of the section (if I'm not mistaken). The following completely expands the given name with an \edef expansion (this might produce new issues, I'm not sure). It however works for your MWE:
\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xcolor}
\usepackage[explicit]{titlesec}
\usepackage{soul}
\definecolor{titleblue}{HTML}{4a7aa4}
\title{Sections and Chapters}
\author{Educ}
\date{\today}
\newbox\TitleUnderlineTestBox
\newcommand\TitleUnderline[1]
{%
\bgroup
\setbox\TitleUnderlineTestBox\hbox{\colorbox{titleblue}\strut}%
\setul{\dimexpr\dp\TitleUnderlineTestBox-.3ex\relax}{.3ex}%
\edef\mytmp{#1}%
\expandafter\ul\expandafter{\mytmp}%
\egroup
}
\newcommand\SectionNumberBox[1]
{%
\colorbox{titleblue}
{%
\makebox[2.5em][c]
{%
\color{white}%
\strut
\csname the#1\endcsname
}%
}%
\TitleUnderline{\ \ \ }%
}
\titleformat{\section}
{\Large\bfseries\sffamily\color{titleblue}}
{\SectionNumberBox{section}}
{0pt}
{\TitleUnderline{#1}}
\titleformat{\subsection}
{\large\bfseries\sffamily\color{titleblue}}
{\SectionNumberBox{subsection}}
{0pt}
{\TitleUnderline{#1}}
\begin{document}
\maketitle
\section{The First Section}
\begin{thebibliography}{}
\bibitem{}
\end{thebibliography}
\end{document}
Alternative
The following defines an alternative style, that formats unnumbered sections similar to the numbered ones (it uses report and also defines \chapter to be matching, just remove that and you're good to go with article). It also removes the unnecessary repetition of calling \titleformat for each sectioning level by defining a shorthand macro.
\documentclass{report}
\usepackage[utf8]{inputenc}
\usepackage{xcolor}
\usepackage[explicit]{titlesec}
\usepackage{soul}
\definecolor{titleblue}{HTML}{4a7aa4}
\title{Sections and Chapters}
\author{Educ}
\date{\today}
\newlength\SectionNumberBoxPad
\newcommand\TitleUnderline[1]
{%
\begingroup
\setul{\dimexpr\dp\strutbox+\fboxsep-.3ex\relax}{.3ex}%
\edef\mytmp{#1}%
\expandafter\ul\expandafter{\mytmp}%
\endgroup
}
\newcommand\SectionNumberBoxSep{\TitleUnderline{\ \ \ }}
\newcommand\UnnumberedSectionNumberBox
{%
\colorbox{titleblue}
{\strut\hspace{\dimexpr\ht\strutbox+\dp\strutbox}}%
\SectionNumberBoxSep
}
\newcommand\SectionNumberBox[1]
{%
\colorbox{titleblue}
{%
\begingroup
\color{white}%
\setlength\SectionNumberBoxPad{.5\dimexpr\ht\strutbox+\dp\strutbox}%
\hspace{\SectionNumberBoxPad}%
\strut\csname the#1\endcsname
\hspace{\SectionNumberBoxPad}%
\endgroup
}%
\SectionNumberBoxSep
}
\makeatletter
\newcommand\setuptitleformat[2]
{%
\expandafter\setuptitleformat@\csname #1\endcsname{#1}{#2}%
}
\newcommand\setuptitleformat@[3]
{%
\titleformat{#1}
{#3\bfseries\sffamily\color{titleblue}}
{\SectionNumberBox{#2}}
{0pt}
{\TitleUnderline{##1}}%
\titleformat{name=#1,numberless}
{#3\bfseries\sffamily\color{titleblue}}
{\UnnumberedSectionNumberBox}
{0pt}
{\TitleUnderline{##1}}%
}
\makeatother
\setuptitleformat{chapter}{\LARGE}
\setuptitleformat{section}{\Large}
\setuptitleformat{subsection}{\large}
\setuptitleformat{subsubsection}{}
\setcounter{secnumdepth}{3}
\setcounter{chapter}{10}
\begin{document}
\maketitle
\chapter{The First Chapter}
\section{The First Section}
\subsection{The First Subsection}
\subsubsection{The First Subsubsection}
Some text
\begin{thebibliography}{}
\bibitem{}
\end{thebibliography}
\end{document}

Alternative 2
This version should look identical in print, but the display of the PDF depends on the viewer (xpdf can be trusted, though; this is because using ulem the lines are built using small segments which might lead to aliasing issues on screens). It breaks automatic hyphenation, but remains line breakable. It uses the ulem package instead of the soul package. ulem should be robuster, and therefore the \edef expansion is not needed with it.
\documentclass{report}
\usepackage[utf8]{inputenc}
\usepackage{xcolor}
\usepackage[explicit]{titlesec}
\usepackage[normalem]{ulem}
\definecolor{titleblue}{HTML}{4a7aa4}
\title{Sections and Chapters}
\author{Educ}
\date{\today}
\newlength\SectionNumberBoxPad
\newlength\TitleUnderlineSeg
\setlength\TitleUnderlineSeg{.05pt}
\newcommand\TitleUnderline[1]
{%
\bgroup
\markoverwith
{%
\rule
[-\dimexpr\dp\strutbox+\fboxsep\relax]
{\TitleUnderlineSeg}
{.3ex}%
}%
\ULon{#1}%
}
\newcommand\SectionNumberBoxSep
{%
\hspace{-\TitleUnderlineSeg}%
\TitleUnderline
{\hspace{\TitleUnderlineSeg}\hspace{\SectionNumberBoxPad}}%
}
\newcommand\UnnumberedSectionNumberBox
{%
\begingroup
\setlength\SectionNumberBoxPad{.5\dimexpr\ht\strutbox+\dp\strutbox}%
\colorbox{titleblue}
{\strut\hspace{2\SectionNumberBoxPad}}%
\SectionNumberBoxSep
\endgroup
}
\newcommand\SectionNumberBox[1]
{%
\begingroup
\setlength\SectionNumberBoxPad{.5\dimexpr\ht\strutbox+\dp\strutbox}%
\colorbox{titleblue}
{%
\begingroup
\color{white}%
\hspace{\SectionNumberBoxPad}%
\strut\csname the#1\endcsname
\hspace{\SectionNumberBoxPad}%
\endgroup
}%
\SectionNumberBoxSep
\endgroup
}
\makeatletter
\newcommand\setuptitleformat[2]
{%
\expandafter\setuptitleformat@\csname #1\endcsname{#1}{#2}%
}
\newcommand\setuptitleformat@[3]
{%
\titleformat{#1}
{#3\bfseries\sffamily\color{titleblue}}
{\SectionNumberBox{#2}}
{0pt}
{\TitleUnderline{##1}}%
\titleformat{name=#1,numberless}
{#3\bfseries\sffamily\color{titleblue}}
{\UnnumberedSectionNumberBox}
{0pt}
{\TitleUnderline{##1}}%
}
\makeatother
\setuptitleformat{chapter}{\LARGE}
\setuptitleformat{section}{\Large}
\setuptitleformat{subsection}{\large}
\setuptitleformat{subsubsection}{}
\setcounter{secnumdepth}{3}
\setcounter{chapter}{10}
\begin{document}
\maketitle
\chapter{The First Chapter}
\section{The First Section}
\subsection{The First Subsection}
\subsubsection{The First Subsubsection}
Some text
\section
{Really long section name that is really long, so long it takes two rows}
\begin{thebibliography}{}
\bibitem{}
\end{thebibliography}
\end{document}

The lines do align vertically with the box of the section numbers (checked with xpdf at 2000% magnification), the mismatch is due to Gimp's rasterization.
\end{thebibliography}. – Skillmon Feb 22 '19 at 15:16\section*{Foo}does work. – Skillmon Feb 22 '19 at 15:20