How to expand values stored in a token defined by newtoks

Question

In this project, the directory structure and file names are based on three parameters, with there being one exception to both.

Directory Path: \GetTexFilePath{}{}{}

The directory path is based on the top level directory (#1), the first parameter passed to the macro.

Normally the path is of form ../#1/xxx#2/#3/, except when #1=\SpecialCase in which case, the xxx is not part of the path.

File Names: \GetTexFileName{}{}{}

The file name is dependent on the current value of the token: \the\CurrentTokenValue.

The files names are of the form #1-#2-#3, except for one special case when \the\CurrentTokenValue=\SpecialCase, in which case, the file name is just #3.

---

Below I am using the expl3 solution adapted from Text being output when none should be.

As requested by @egreg, I have added a section at the beginning to test the various cases and have added the expected output in the comments. Have also added red color to the failing output, and blue color to the correct output.

The first two sections following the above list, work just fine, but the third case where the file name depends on the value of \the\CurrentTokenValue fails.

Also note that in section 3, #1=\the\CurrentTokenValue and the directory name correctly does not have the xxx in it (which is correct). But the file name did not get shortened. Section 3 yields:

../special/02/z/special-02-z

but the correct value for Case 3 should be

../special/02/z/z

Questions:

1. How do I adjust \GetTexFileName so that the file name depends on the token?
2. Why did this expansion issue of tokens not have a problem when used in the file path, just the file name?

I need to use the concatenated output of the path and file name macros (with a / in between) to be able to to check \IfFileExists.

Answer:

• I have confirmed that changing the \def\SpecialValue{special} to \protected\def\SpecialValue{special} as per @egreg's answer works for all these cases. My concern with that is I don't know if that will have any other effect on the usage of \SpecialValue, and hence a solution local to \GetTexFilePath and \GetTexFileName would be preferable.
• Bruno's suggestion to add \expandafter\empty before the first parameter to \str_if_eq:xxTF works for this test cast, but fails in my actual usage for some reason.

Notes:

• The complaint about the file not being found is not the problem.
• All working cases are in blue.
• All failing cases are in red.

Code:

\documentclass{article}
\usepackage{xcolor}
\usepackage{xparse}
\def\SpecialValue{special}
\newtoks{\CurrentTokenValue}
\ExplSyntaxOn
% Determine path: Always #1/xxx#2/#3 unless #1=\SpecialValue
\NewDocumentCommand{\GetTexFilePath}{m m m}{%
    \str_if_eq:xxTF{#1}{\SpecialValue}{../#1/#2/#3}{../#1/xxx#2/#3}%
}%
% Determine file name: Always #1-#2-#3 unless \the\CurrentTokenValue=\SpecialValue.
\NewDocumentCommand{\GetTexFileName}{m m m}{%
    \str_if_eq:xxTF{\the\CurrentTokenValue}{\SpecialValue}{#3}{#1-#2-#3}%
}%
\ExplSyntaxOff
\newcommand*{\GetFileNameWithPath}[3]{%
    \GetTexFilePath{#1}{#2}{#3}/%
    \GetTexFileName{#1}{#2}{#3}%
}%
\newcommand*{\TestFile}[2][red]{%
    \noindent%
    Token Value = \the\CurrentTokenValue%
    \par\noindent%
    \IfFileExists{#2}{%
        \textcolor{#1}{File \FileWithPath\ found.}%
    }{%
        \textcolor{#1}{File \FileWithPath\ not found.}%
    }%
}%
\begin{document}
\CurrentTokenValue={NonSpecial}%
\color{blue}% These work (up until the change to red color)
1a. \GetTexFilePath{A}{B}{C}             \par%          ../A/xxxB/C           Note: xxx
2a. \GetTexFilePath{\SpecialValue}{B}{C} \par%          ../special/B/C        Note: no xxx
3a. \GetTexFilePath{\the\CurrentTokenValue}{B}{C} \par% ../NonSpecial/xxxB/C  Note: xxx
%   ----------------
4a. \GetTexFileName{A}{B}{C}             \par%          A-B-C
5a. \GetTexFileName{\SpecialValue}{B}{C} \par%          special-B-C
6a. \GetTexFileName{\the\CurrentTokenValue}{B}{C} \par% NonSpecial-B-C
\smallskip\hrule\smallskip
% Changing the value of \CurrentTokenValue to this one special value, 
% should only change the file name, and should not effect the 
% file path.  Absolutely no reason why the first two here should be 
% any different than above, but included here for completeness.
\CurrentTokenValue={\SpecialValue}%
1b. \GetTexFilePath{A}{B}{C}             \par%          ../A/xxxB/C           Note: xxx
2b. \GetTexFilePath{\SpecialValue}{B}{C} \par%          ../special/B/C        Note: no xxx
\color{red}% These next 4 fail
3b. \GetTexFilePath{\the\CurrentTokenValue}{B}{C} \par% ../special/B/C        ** FAILS **
%   ---------------
4b. \GetTexFileName{A}{B}{C}             \par%          C                     ** FAILS **
5b. \GetTexFileName{\SpecialValue}{B}{C} \par%          C                     ** FAILS **
6b. \GetTexFileName{\the\CurrentTokenValue}{B}{C} \par% C                     ** FAILS **
\color{black}
\bigskip\hrule\hrule\bigskip
\CurrentTokenValue={NonSpecial}
\section{Normal Case: Works}
Neither the first parameter, nor the \verb|\the\CurrentTokenValue| is  \verb|\SpecialValue| so should have \verb|xxx| in the path, and a file name with two dashes:
\edef\FileWithPath{\GetFileNameWithPath{NonSpecial}{02}{z}}%
\TestFile[blue]{\FileWithPath}
\section{Top directory is special case: Works}
The first parameter is \verb|\SpecialValue|, but not \verb|\the\CurrentTokenValue| so should \textbf{not} have \verb|xxx| in the path, and a file name with two dashes:
\edef\FileWithPath{\GetFileNameWithPath{\SpecialValue}{02}{z}}%
\TestFile[blue]{\FileWithPath}
\section{Both Top directory and TokenValue are special cases}
Since both the first parameter is \verb|\SpecialValue| and \verb|\the\CurrentTokenValue| are a special case, we should \textbf{not} have \verb|xxx| in the path, and a file name should be short:
\CurrentTokenValue={\SpecialValue}
\edef\FileWithPath{\GetFileNameWithPath{\the\CurrentTokenValue}{02}{z}}%
\TestFile{\FileWithPath}
\bigskip
\noindent
Correct result for the above case is
\def\FileWithPath{../special/02/z/z}%
\TestFile[blue]{\FileWithPath}
\end{document}

Answer 1

I'd go with a token list, but the simple fix

\protected\def\SpecialValue{special}

seems to give the required output: in section 3 I get

Token Value = special
File ../special/02/z/z not found.

Correct value for this case is
Token Value = special
File ../special/02/z/z not found.

In the "special" case the comparison is precisely between \SpecialValue and \SpecialValue (not expanded).

A different approach is to avoid token registers:

\ExplSyntaxOn
\tl_new:N \SpecialValue
\tl_set:Nn \SpecialValue {special}
\tl_new:N \l_grill_current_value_tl

\NewDocumentCommand{\CurrentTokenValue}{m}
  {
   \tl_set:Nn \l_grill_current_value_tl { #1 }
  }
\DeclareExpandableDocumentCommand{\GetCurrentTokenValue}{ }
  {
   \tl_use:N \l_grill_current_value_tl
  }
% Determine path: Always #1/xxx#2/#3 unless #1=\SpecialValue
\NewDocumentCommand{\GetTexFilePath}{m m m}
  {
   \str_if_eq:xxTF{#1}{\SpecialValue}{../#1/#2/#3}{../#1/xxx#2/#3}
  }

% Determine file name: Always #1-#2-#3 unless \the\CurrentTokenValue=\SpecialValue.
\NewDocumentCommand{\GetTexFileName}{m m m}
  {
   \str_if_eq:xxTF{\l_grill_current_value_tl}{\SpecialValue}{#3}{#1-#2-#3}%
  }
\ExplSyntaxOff

It's sufficient now to change the syntax \CurrentTokenValue={...} into \CurrentTokenValue{...} and \the\CurrentTokenValue into \GetCurrentTokenValue.