Can't make a list of keys work

Question

I am trying to make my own chapter-heading formatting command, using a key=value approach:

\coolchap[
chapstyle=<chapter-word formatting>,
titlestyle=<chapter-title formatting>,
befchap=<vspace before chapter-word>,
afchap=<vspace after chapter-word>,
aftitle=<vspace after chapter-title>]

But something must be wrong, because it has no effect: I receive no error message, but the values are being just ignored.

Note that the same happens if I define the argument as obligatory instead, then pass the command \coolchap{key1=value1,key2=value2...}.

The auxiliary command itself, i.e. \chapterformat, works.

MWE

\documentclass{report}
\usepackage{etoolbox}
\usepackage{moresize}
\makeatletter
\NewDocumentCommand{\chapterformat}{mmmmm}{%
    \ifdef{\chapter}{%
        \renewcommand{@makechapterhead}[1]{%
            \vspace*{#3\p@}% Vertical Space before "Chapter X" (50)
            {\parindent \z@ \raggedright \reset@font
                \ifnum \c@secnumdepth >\m@ne
                {#1 @chapapp{} \thechapter} % Chapter-word formatting.
                \par
                \vskip #4\p@ % Vertical Space after "Chapter X" (20)
                \fi
                {#2 ##1} % Title formatting.
                \par\nobreak
                \vskip #5\p@}} % Vertical Space after Chapter Title (180)
    }{}
}
\makeatother
\ExplSyntaxOn
\cs_generate_variant:Nn \xcoolchap:nnnnn { VVVVV }
\cs_set_eq:NN \xcoolchap:nnnnn \chapterformat
\NewDocumentCommand{\coolchap}{o}
{
    \group_begin:
      \keys_set:nn { coolchap } { #1 }
      \xcoolchap:VVVVV
        \xcoolchap_chapstyle
        \xcoolchap_titlestyle
        \xcoolchap_befchap
        \xcoolchap_afchap
        \xcoolchap_aftitle
    \group_end:
}
\keys_define:nn { coolchap }
{
    chapstyle .tl_set:N = \xcoolchap_chapstyle,
    titlestyle .tl_set:N = \xcoolchap_titlestyle,
    befchap .tl_set:N = \xcoolchap_befchap,
    afchap .tl_set:N =  \xcoolchap_afchap,
    aftitle .tl_set:N = \xcoolchap_aftitle,
    chapstyle .initial:n = \HUGE\bfseries,
    titlestyle .initial:n = \Huge\bfseries,
    befchap .initial:n = 50,
    afchap .initial:n = 20,
    aftitle .initial:n = 180,
}
\ExplSyntaxOff
\begin{document}
\coolchap[aftitle=360,befchap=100,afchap=40,chapstyle=\HUGE\itshape]

\chapter{essai}

XXX



\end{document}

Answer 1

A few notes on your code:

• use \par within the group in which font changes are applied, so instead of {\huge Title string}\par use {\huge Title string\par} or else the interline spacing will end up wrong.
• you had a few unwanted spaces in your definition, note that a space after #1 in your definition is not ignored, even if #1 will be \huge or something like that at use time.
• your variables aren't following expl3 naming conventions, I suggest you correct this (those conventions are there for a reason, even if there is no sane way to make TeX enforce them).

I didn't correct all of those in each code block, so don't just copy these 1:1 but pay attention where you need to fix them.

---

The issue is that you redefine \@makechapterhead inside of a group and the definition is lost at its end. You have to make the redefinition after \group_end:, you could do this like the following. Also, your \coolchap macro does throw an error if you omit the optional argument, you should use O{} instead of o like in the following.

\documentclass{report}
\usepackage{etoolbox}
\usepackage{moresize}
\makeatletter
\NewDocumentCommand{\chapterformat}{mmmmm}{%
    \ifdef{\chapter}{%
        \renewcommand{@makechapterhead}[1]{%
            \vspace*{#3\p@}% Vertical Space before "Chapter X" (50)
            {\parindent \z@ \raggedright \reset@font
                \ifnum \c@secnumdepth >\m@ne
                {#1 @chapapp{} \thechapter} % Chapter-word formatting.
                \par
                \vskip #4\p@ % Vertical Space after "Chapter X" (20)
                \fi
                {#2 ##1} % Title formatting.
                \par\nobreak
                \vskip #5\p@}} % Vertical Space after Chapter Title (180)
    }{}
}
\makeatother
\ExplSyntaxOn
\tl_new:N \l__xcoolchap_redefinition_tl
\NewDocumentCommand{\coolchap}{O{}}
{
    \group_begin:
      \keys_set:nn { coolchap } { #1 }
      \tl_set:Nx \l__xcoolchap_redefinition_tl
        {
          \exp_not:N \chapterformat
            { \exp_not:V \l__xcoolchap_chapstyle_tl }
            { \exp_not:V \l__xcoolchap_titlestyle_tl }
            { \exp_not:V \l__xcoolchap_befchap_tl }
            { \exp_not:V \l__xcoolchap_afchap_tl }
            { \exp_not:V \l__xcoolchap_aftitle_tl }
        }
      \exp_last_unbraced:NV
    \group_end:
    \l__xcoolchap_redefinition_tl
}
\keys_define:nn { coolchap }
{
    chapstyle .tl_set:N = \l__xcoolchap_chapstyle_tl,
    titlestyle .tl_set:N = \l__xcoolchap_titlestyle_tl,
    befchap .tl_set:N = \l__xcoolchap_befchap_tl,
    afchap .tl_set:N =  \l__xcoolchap_afchap_tl,
    aftitle .tl_set:N = \l__xcoolchap_aftitle_tl,
    chapstyle .initial:n = \HUGE\bfseries,
    titlestyle .initial:n = \Huge\bfseries,
    befchap .initial:n = 50,
    afchap .initial:n = 20,
    aftitle .initial:n = 180,
}
\ExplSyntaxOff
\begin{document}
\coolchap[aftitle=36,befchap=10,afchap=40,chapstyle=\HUGE\itshape]

\chapter{essai}

XXX



\end{document}

---

Disclaimer: I'm the author of expkv-cs

Note that this is a lot easier with expkv-cs since it doesn't work with assignments and hence has no such grouping issues:

\documentclass{report}
\usepackage{etoolbox}
\usepackage{moresize}
\usepackage{expkv-cs}
\makeatletter
\NewDocumentCommand{\chapterformat}{mmmmm}{%
    \ifdef{\chapter}{%
        \renewcommand{@makechapterhead}[1]{%
            \vspace*{#3\p@}% Vertical Space before "Chapter X" (50)
            {\parindent \z@ \raggedright \reset@font
                \ifnum \c@secnumdepth >\m@ne
                {#1 @chapapp{} \thechapter} % Chapter-word formatting.
                \par
                \vskip #4\p@ % Vertical Space after "Chapter X" (20)
                \fi
                {#2 ##1} % Title formatting.
                \par\nobreak
                \vskip #5\p@}} % Vertical Space after Chapter Title (180)
    }{}
}
\NewDocumentCommand\coolchap{O{}}{\coolchapKV{#1}}
\ekvcSplitAndForward\coolchapKV\chapterformat
  {
     chapstyle  = \HUGE\bfseries
    ,titlestyle = \Huge\bfseries
    ,befchap    = 50
    ,afchap     = 20
    ,aftitle    = 180
  }
\makeatother
\begin{document}
\coolchap[aftitle=36,befchap=10,afchap=40,chapstyle=\HUGE\itshape]

\chapter{essai}

XXX



\end{document}

---

Output of both codes:

---

A different approach could be to just redefine \chapterformat and use your variables inside of that redefinition. That way you don't have to use any expansion tricks whatsoever and continue to use l3keys:

\documentclass{report}
\usepackage{etoolbox}
\usepackage{moresize}
\makeatletter
\ExplSyntaxOn
\ifdef{\chapter}{%
  \renewcommand{@makechapterhead}[1]
    {%
      \vspace*{\l__xcoolchap_befchap_tl\p@}% Vertical Space before "Chapter X" (50)
      \begingroup
        \parindent \z@ \raggedright \reset@font
        \ifnum \c@secnumdepth >\m@ne
          {\l__xcoolchap_chapstyle_tl @chapapp{} \thechapter\par}% Chapter-word formatting.
          \vskip \l__xcoolchap_afchap_tl\p@ % Vertical Space after "Chapter X" (20)
        \fi
        {\l__xcoolchap_titlestyle_tl #1\par}% Title formatting.
        \nobreak
        \vskip \l__xcoolchap_aftitle_tl\p@
      \endgroup
    }% Vertical Space after Chapter Title (180)
}{}
\makeatother
\NewDocumentCommand{\coolchap}{O{}}
  { \keys_set:nn { coolchap } { #1 } }
\keys_define:nn { coolchap }
{
    chapstyle .tl_set:N = \l__xcoolchap_chapstyle_tl,
    titlestyle .tl_set:N = \l__xcoolchap_titlestyle_tl,
    befchap .tl_set:N = \l__xcoolchap_befchap_tl,
    afchap .tl_set:N =  \l__xcoolchap_afchap_tl,
    aftitle .tl_set:N = \l__xcoolchap_aftitle_tl,
    chapstyle .initial:n = \HUGE\bfseries,
    titlestyle .initial:n = \Huge\bfseries,
    befchap .initial:n = 50,
    afchap .initial:n = 20,
    aftitle .initial:n = 180,
}
\ExplSyntaxOff
\begin{document}
\coolchap[aftitle=36,befchap=10,afchap=40,chapstyle=\HUGE\itshape]

\chapter{essai}

XXX



\end{document}