I created a class file named class.cls as follows:
\NeedsTeXFormat{LaTeX2e}
\ProvidesClass{class}
\LoadClass{report}
\RequirePackage{indentfirst}
\DeclareOption{withinchap}{%
\newcounter{exercounter}[chapter]%
\setcounter{exercounter}{0}}
\DeclareOption{withinsec}{%
\newcounter{exercounter}[section]%
\setcounter{exercounter}{0}}
\DeclareOption*{\PassOptionsToClass{\CurrentOption}{report}}
\ExecuteOptions{withinchap}
\ProcessOptions\relax
\newenvironment{exercise}%
{\stepcounter{exercounter}\par\textbf{\theexercounter.}}%
{\ignorespacesafterend}
What I want to achieve is to settle a pair of class options withinchap(default) and withinsec. If using withinchap(default) option, the exercise counter exercounter is reset to zero when a new chapter starts. If using withinsec option, the exercise counter exercounter is reset to zero when a new section starts. But when I use withinsec option in the test.tex file as follows:
\documentclass[withinsec]{class}
\usepackage{showframe}
\begin{document}
\chapter{AAAAA}
\section{aaaaa}
\begin{exercise}
This is an exercise.
\end{exercise}
\begin{exercise}
This is another exercise.
\end{exercise}
\begin{exercise}
This is the third exercise.
\end{exercise}
\section{bbbbb}
\begin{exercise}
This is the fourth exercise in chapter one.
\end{exercise}
\begin{exercise}
This is the fifth exerecise in chapter one.
\end{exercise}
\end{document}
After running pdflatex test in windows terminal, I got an error message
! LaTeX Error: Command \c@exercounter already defined.
Or name \end... illegal, see p.192 of the manual.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.18 \ProcessOptions\relax
?
! Emergency stop.
...
l.18 \ProcessOptions\relax
! ==> Fatal error occurred, no output PDF file produced!
Transcript written on test.log.
Type H <return> for immediate help.
...
l.18 \ProcessOptions\relax
?
I want to know the reason of the error.


withinsec, gets processed, the default option, i.e.,withinchap, has already been processed, and the counter namedexercounterhas been set up. That is why\newcounter{exercounter}[section]generates an error message. What you need, then, is\renewcounter{exercounter}[section]. That, or go with @egreg's answer and use the\counterwithinapproach. – Mico Jan 30 '22 at 10:00