Many thanks once again to David Carlisle for providing the answer! Based on it I have made some improvements.
Text does not collide quite so easily anymore. (though not perfect yet). And formatting is nicer.
In case anyone else is interested, here's the code:
\documentclass[10pt, a4paper, twocolumn, twoside, openright]{memoir}
\usepackage{etoolbox}
\usepackage{ragged2e}
\usepackage{qrcode} % for qr code example
\usepackage{lipsum} % for blindtext
\usepackage{fontawesome} % for symbols
\newcounter{mySideNoteCounter}
\makeatletter
\def\mySideNote#1{{%
\refstepcounter{mySideNoteCounter}\textsuperscript{\themySideNoteCounter}
\if@firstcolumn
\marginpar{\RaggedRight\fussy\footnotesize\textit{\textcolor{red}{\faCaretLeft~}\textsuperscript{\themySideNoteCounter}\\ #1}}
\else
\marginpar{\textsuperscript{\themySideNoteCounter}\RaggedLeft\fussy\footnotesize\textit{\textcolor{red}{~\faCaretRight}\\ #1}}
\fi
}}
\makeatother
\makeatletter
\patchcmd\@addmarginpar{\if@firstcolumn}{\unless\if@firstcolumn}{}{}
\makeatother
\setlength\columnsep{3.5cm}
\setlength\marginparsep{0.2cm}
\setlength\marginparwidth{3cm}
%\makeatletter
%\newcommand{\LorR}{%
% \if@firstcolumn
% LEFT%
% \else
% RIGHT%
% \fi
%}
%\makeatother
\begin{document}
\section{Section}
\subsection{Subsection}
TEXT\mySideNote{Information.\faBarChart}
\lipsum[1] Text text TEXT\mySideNote{Info}
\subsection{Subsection}
%\LorR
TEXT\mySideNote{A little note!}
Some text is written here. It is not very interesting. Just for testing. Simply ignore it.
TEXTTEXT\mySideNote{Here is a side note! Some interesting information here}
\lipsum[1]
\mySideNote{Look:\\\qrcode{https://www.ctan.org/tex-archive/macros/latex/contrib/qrcode?lang=en}} \lipsum[2]
HELLO\mySideNote{Some more information here!}
\lipsum[1]
TEXT\mySideNote{Look? There's another marginnote!}
\lipsum[5-8]
MORE TEXT \mySideNote{And again a margin note! Extraordinary, even some qr code!\\\qrcode{https://www.ctan.org/tex-archive/macros/latex/contrib/qrcode?lang=en}}
\lipsum[1]
\end{document}
This then gives the following result:

Any ideas on how to implement the separating lines are welcome!
I couldn't find an answer to that one.
Also, it would be interesting to know how to arrange it as Frank Mittelbach says. That is, to use the middle margin for footnotes. The top ones for the left column, the bottom ones for the right column.
P.S. I was user145210 who originally posted the question. When I finished registration it did not update the account. Don't know what happened there.