I am unsure how your CSV file looks like, but guessing from what you describe, an easy approach could be as follows: I understand that you got a CSV file with a content similar to this:
Epoch, Test_loss, Train_loss, Test_metric
1, 0.02, 0.03, 0.025
1000, 0.01, 0.005, 0.015
...
In this case, you can simply parse this by adding an \addplot macro for each of the columns that you want to plot, let x be the Epoch column and y the column with the relevant values:
\documentclass[tikz, border=1cm]{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}
% your CSV file:
\begin{filecontents}{data.csv}
Epoch, Test_loss, Train_loss, Test_metric
1, 0.02, 0.03, 0.025
1000, 0.01, 0.005, 0.015
\end{filecontents}
% =====
\begin{document}
\begin{tikzpicture}
\begin{semilogyaxis}[
xmin=0, xmax=1e4,
ymin=1e-4, ymax=1e-1,
scaled x ticks=false,
xlabel={Number of Epochs},
ylabel={Normalized MSE: $\log_{10} (e)$},
label style={font=\bfseries\boldmath},
tick label style={font=\bfseries\boldmath},
]
\addplot[scatter, no marks, draw=blue]
table [x=Epoch, y=Test_loss, col sep=comma] {data.csv};
\addplot[scatter, no marks, draw=red]
table [x=Epoch, y=Train_loss, col sep=comma] {data.csv};
\addplot[scatter, no marks, draw=orange]
table [x=Epoch, y=Test_metric, col sep=comma] {data.csv};
\end{semilogyaxis}
\end{tikzpicture}
\end{document}

Of course, you don't need to include the CSV data in your TeX file. Just place the CSV file in the same directory where your TeX file is stored, and write, e.g., \addplot[scatter, no marks, draw=blue] table [...] {<file name>};. You can just ignore the stuff between \begin{filecontents}{data.csv} and \end{filecontents} (including these two lines) in this case, since this is just to make the above example compilable.
Sorry, I cannot really reproduce the problems you have. I adjusted the above example with the data you provided:
The file 3N5H.csv (note that there should probably be no empty lines in the CSV file, because these would be interpreted as (empty) data points):
Epochs,Test_loss,Train_loss
0,0.203095777,0.234712227
1,0.202614659,0.234308232
2,0.202137079,0.233968432
The TeX file:
\documentclass[tikz, border=1cm]{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}
\begin{document}
\begin{tikzpicture}
\begin{semilogyaxis}[
% xmin=0, xmax=1e4, % The output would not be visible otherwise
% ymin=1e-4, ymax=1e-1, % The output would not be visible otherwise
scaled x ticks=false,
enlargelimits,
xlabel={Number of Epochs},
ylabel={Normalized MSE: $\log_{10} (e)$},
label style={font=\bfseries\boldmath},
tick label style={font=\bfseries\boldmath},
]
\addplot[scatter, no marks, draw=blue] table [x=Epochs, y=Test_loss, col sep=comma] {3N5H.csv};
\addplot[scatter, no marks, draw=red] table [x=Epochs, y=Train_loss, col sep=comma] {3N5H.csv};
\end{semilogyaxis}
\end{tikzpicture}
\end{document}
Result:

TeX capacity exceeded, sorry [main memory size=5000000].– Formal_this Feb 06 '23 at 21:22