Try the following version of cleveref.4ht:
% cleveref.4ht (2018-06-08-16:51), generated from tex4ht-4ht.tex
% Copyright 2018 TeX Users Group
%
% This work may be distributed and/or modified under the
% conditions of the LaTeX Project Public License, either
% version 1.3c of this license or (at your option) any
% later version. The latest version of this license is in
% http://www.latex-project.org/lppl.txt
% and version 1.3c or later is part of all distributions
% of LaTeX version 2005/12/01 or later.
%
% This work has the LPPL maintenance status "maintained".
%
% The Current Maintainer of this work
% is the TeX4ht Project <http://tug.org/tex4ht>.
%
% If you modify this program, changing the
% version identification would be appreciated.
\immediate\write-1{version 2018-06-08-16:51}
\def\refstepcounter@noarg#1{%
\cref@old@refstepcounter{#1}%
\cref@constructprefix{#1}{\cref@result}%
\@ifundefined{cref@#1@alias}%
{\def\@tempa{#1}}%
{\def\@tempa{\csname cref@#1@alias\endcsname}}%
\protected@xdef\cref@currentlabel{%
[\@tempa][\arabic{#1}][\cref@result]%
\csname p@#1\endcsname\csname the#1\endcsname}
}%
\def\refstepcounter@optarg[#1]#2{%
\cref@old@refstepcounter{#2}%
\cref@constructprefix{#2}{\cref@result}%
\@ifundefined{cref@#1@alias}%
{\def\@tempa{#1}}%
{\def\@tempa{\csname cref@#1@alias\endcsname}}%
\protected@edef\cref@currentlabel{%
[\@tempa][\arabic{#2}][\cref@result]%
\csname p@#2\endcsname\csname the#2\endcsname}}%
\ifdefined\@firstoffive\else%
\def\@firstoffive#1#2#3#4#5{#1}
\fi
\def\:tempa#1#2{\bgroup%
\def\rEfLiNK##1##2{\Link{##1}{}}%
\expandafter\expandafter\expandafter\@firstoffive\csname r@#2\endcsname{}{}{}{}{}%
\o:@@@setcref:{#1}{#2}%
\EndLink\egroup%
}%
\HLet\@@@setcref=\:tempa
\Hinput{cleveref}
\endinput
It patches the \@@@setcref command, which is used for the printing of the references. The patch executes the current reference, in order to get the link. Because of the reference structure used by tex4ht when the Hyperref package is loaded is a bit complicated, we need to use some tricks like the \@firstofone command etc. It works with and without Hyperref though.
The solution supports even multiple references in one \cref command, as shown in a modified example:
\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc} % may use latin1 instead of utf8
\usepackage[]{hyperref}
\usepackage{cleveref}
\begin{document}
\section{testsection}
\label{testlabel}
Go to \cref{testlabel,secondsectioon}
\section{second}
\label{secondsectioon}
Go to \ref{testlabel}
\end{document}
This is the result:

There are two different links used in \cref, as is visible from the HTML code:
<h3 class="sectionHead"><span class="titlemark">1 </span> <a
id="x1-10001"></a>testsection</h3>
<!--l. 11--><p class="noindent" >Go to <a
href="#x1-10001">sections 1</a><a
href="#x1-20002"> and 2</a>
</p>
<h3 class="sectionHead"><span class="titlemark">2 </span> <a
id="x1-20002"></a>second</h3>
<!--l. 15--><p class="noindent" >Go to <a
href="#x1-10001">1<!--tex4ht:ref: testlabel --></a>
</p>