If you'd not use package caption redefining \thefigure would not result in “Figure 3 (continued)” but “Figure 3 (continued):” (please note the colon at the end). This does not happen with package caption, but nevertheless I would suggest to define a command \ContinuedCaption, that changes the labelformat instead of redefining \thefigure:
\documentclass{article}
\usepackage{caption}
\newcommand*{\ContinuedCaption}{%
\ContinuedFloat% Because \ContinuedCaption makes only sense inside continued floats
\begingroup% only needed, if \ContinuedCaption should be mixed with standard \caption in the same float
\captionsetup{labelsep=space}%
\caption[]{(continued)}%
\endgroup% to end the group started with \begingroup and therefore the effect of \captionsetup
}
\usepackage{mwe}
\begin{document}
\listoffigures
\section{Test}
\blindtext
\begin{figure}[ht!]
\centering
\includegraphics{example-image-a}
\caption{This is an example caption for the first part of the example image}
\end{figure}
\blindtext
\begin{figure}
\centering
\includegraphics{example-image-b}
\ContinuedCaption
\end{figure}
\end{document}
I've encapsulated the change of labelformat to a local group. So the effect is limited to the usage of \ContinuedCaption.
The continued figure now would be:

Using an empty optional argument for \caption[]{(continued)} as shown, does prevent adding an entry into the list of figures. If you'd remove it, there would be an entry “Figure 1 (continued)” also in the list of figures. You could also add an optional argument to the definition, used only for an optional entry to the list of figures:
\newcommand*{\ContinuedCaption}[1][]{%
\ContinuedFloat% Because \ContinuedCaption makes only sense inside continued floats
\begingroup
\captionsetup{labelsep=space}%
\caption[{#1}]{(continued)}%
\endgroup
}
or
\newcommand*{\ContinuedCaption}[1][\emph{(continued)}]{%
\ContinuedFloat% Because \ContinuedCaption makes only sense inside continued floats
\begingroup
\captionsetup{labelsep=space}%
\caption[{#1}]{(continued)}%
\endgroup
}
However, if you insist in changing \thefigure, you could add this to the definition of \ContinuedCaption:
\documentclass{article}
\usepackage{caption}
\newcommand{\ContinuedFigureCaption}{%
\ContinuedFloat% Because \ContinuedCaption makes only sense inside continued floats
\begingroup
\NewCommandCopy\OriginalTheCaption\thefigure
\renewcommand\thefigure{\OriginalTheCaption~(continued)}%
% \captionsetup{labelsep=space}% not needed because package caption does this with empty captions automatically
\caption[]{}% empty optional argument needed (see below)
\endgroup
}
\usepackage{mwe}
\begin{document}
\listoffigures
\section{Test}
\blindtext
\begin{figure}[ht!]
\centering
\includegraphics{example-image-a}
\caption{This is an example caption for the first part of the example image}
\end{figure}
\blindtext
\begin{figure}
\centering
\includegraphics{example-image-b}
\ContinuedFigureCaption
\end{figure}
\end{document}
However, this definition is less universal, because it cannot be used, e.g., for tables without changes. Without the optional argument [] for \caption, it also would be problematic for the list of figures, because now “(continued)” is part if the number and the space reserved for numbers in the list of figures is limited. So it would result in an unwanted overlapping of the text and the dots:

So you would also have to redefine \l@figure, if you want (optional) entries to the list of figures. But than you would have a large gap between the number and the text also for main figure entries. So I would not recommend to use a redefinition of \thefigure at least if you want entries for the continues to the list of figures.
The first suggestion is much more general, because it can be used not only for figure but also for table (and other floats). And it does not result in issues with the entries to the list of figures or list of tables.
Note: \NewCommandCopy needs a to old LaTeX kernel. If you are using an old LaTeX, you could replace it (in this case) by \let. But I would suggest to update LaTeX instead.
\begingroup .... \endgroupTeX group? – Mico Feb 01 '23 at 05:13\renewcommand{\thefigure}{use \thefigure here}causes infinite recursion. Before the redefinition, we need to store the original definition in a different name. – Ryo Feb 01 '23 at 05:37\NewCommandCopy\originalTheFigure\thefigure\renewcommand\thefigure{\originalTheFigure~(continued)}. If you absolutely know the structure of\thefigureis just a\newcommand\thefigure{<stuff>}you can also use\let. Scope the thing with a group, (so instead of your\renewcommandafter the\captionusage put a\begingroupin front of the\NewCommandCopyand an\endgroupafter the\caption). – Skillmon Feb 01 '23 at 06:15\thefigurein most document classes is just\arabic{figure}, what's wrong about\renewcommand{\thefigure}{\arabic{figure} (continued)}, sandwiched between\begingroupand\endgroupdirectives? – Mico Feb 01 '23 at 08:07\begingroup! Above your lat comment, @Skillmon shows a complete solution using\begingroupand\endgroup. I tried the solution and it works. Since he or she didn't post it as an answer, I meant to post it as an answer later. – Ryo Feb 01 '23 at 18:02\thefigurewould not have effects for the caption only, but also for the entry to the list of figures and, e.g., for\label/\ref. In the list of figures the “(continued)” would also be part of the number and therefore needs a redefinition of\l@figurewith a very large number width, that would also influence the entries without “(continued)”. So IMHO at least the continue captions should not create entries to the list of figures. This is why I would prefer another suggestions (see my answer). – cabohah Feb 02 '23 at 07:55\caption{}. – Skillmon Feb 02 '23 at 08:03