Please consider this MWE:
\documentclass{article}
\usepackage{lineno}
\usepackage{hyperref}
% \linenumbers
\begin{document}
\url{https://www.this-is-a-very-long-url.com/this-is-a-very-long-url/this-is-a-very-long-url/this-is-a-very-long-url.html}
\end{document}
Using pdflatex, this results in: 
If we uncomment the \linenumbers line, an additional (small) colored box is drawn before the URL: 
Using xelatex, the box is drawn around the line number: 
What's the reason for this extra box? Is there a way to avoid it?
