Defining scalable "white curly brackets" {| and |} (⦃ and ⦄)

Question

I would like to construct what Unicode calls "white curly brackets" that are in addition scalable. They look like ⦃ (≈{|) and ⦄ (≈|}) and have Unicode codepoints U+2983 and U+2984.

unicode-math has them as \lBrace and \rBrace, but (1) I don't presently use XeTeX or LuaTeX and (2) their design (shown in unimath-symbols.pdf on pages 3-4) is not to my liking: the brace part is too fat for my taste, and I would like the vertical bar to not leave any protrusions. I am trying to have them visually match the parentheses \llparenthesis, \rrparenthesis, \llbracket, and \rrbracket from stmaryrd.

Right now I am using a makeshift definition, which is sufficient for now, but the symbols don't scale (in general or with \left/\right). Here is sample code:

\documentclass{article}
\usepackage{amsmath}
\usepackage[only,llbracket,rrbracket,llparenthesis,rrparenthesis]{stmaryrd} % for comparison with 4 similar parenthesis symbols
\usepackage{accsupp} % for ensuring the right Unicode codepoint upon pasting

\newcommand*{\llbrace}{%
  \BeginAccSupp{method=hex,unicode,ActualText=2983}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char102}%
    \mathchoice{\mkern-4.05mu}{\mkern-4.05mu}{\mkern-4.3mu}{\mkern-4.8mu}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char106}%
  \EndAccSupp{}%
}
\newcommand*{\rrbrace}{%
  \BeginAccSupp{method=hex,unicode,ActualText=2984}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char106}%
    \mathchoice{\mkern-4.05mu}{\mkern-4.05mu}{\mkern-4.3mu}{\mkern-4.8mu}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char103}%
  \EndAccSupp{}%
}


\begin{document}

\noindent
ok:
\(\displaystyle\llbrace abc \rrbrace\)
\(\textstyle\llbrace abc \rrbrace\)
\(\scriptstyle\llbrace abc \rrbrace\)
\(\scriptscriptstyle\llbrace abc \rrbrace\)

\noindent
ok:
\(\displaystyle \llbrace \frac{a}{b} \rrbrace\)
\(\textstyle\llbrace \frac{a}{b} \rrbrace\)
\(\scriptstyle\llbrace \frac{a}{b} \rrbrace\)
\(\scriptscriptstyle\llbrace \frac{a}{b} \rrbrace\)

\noindent
incorrectly scaled (\verb|\scriptsize|): {\scriptsize\(\llbrace abc \rrbrace\)} (large magnification required)

\noindent
incorrectly scaled (\verb|\tiny|): {\tiny\(\llbrace abc \rrbrace\)}

\noindent
\verb|\left| and \verb|\right| not possible before \verb|\llbrace| and \verb|\rrbrace|: \\
\indent \(\llbrace \left\llbracket\left\{\left[(a) \cdot bc^{2^3}\right]^4\right\}^5\right\rrbracket \rrbrace\)
% Note: This is also not possible with \llparenthesis and \rrparenthesis from stmaryrd.

\noindent
for comparison: \(\llparenthesis x \rrparenthesis, \llbracket x \rrbracket, \llbrace x \rrbrace; (x), [x], \{x\}\)

\end{document}

Lines 1 and 2 look just right with the present setup. Lines 3 and 4 demonstrate that my definitions don't scale. Lines 5a/b gives an example of where I would like to be able to use \left and \right. Line 6 juxtaposes different types of parentheses for visual comparison.

Is there an easy way to scale the glyphs I have constructed myself? Or is designing new glyphs the only way? While the vertical part of my glyphs doesn't leave any protrusions, the "brace part being too fat" part of my problem isn't being addressed by my code. With this in mind: I recognize that it might not be that easy to make the brace part of the glyph thinner (though if you can, more power to you), but I think that any solution that addresses the scaling problem for any manually constructed glyph cluster will be a valuable contribution to the community.

---

For a slightly different setup (original example code) with

\RequirePackage{fix-cm}
\documentclass[12pt]{memoir}
\usepackage{fixltx2e}[2006/09/13]
\usepackage{amsmath}
\usepackage{txfonts}
\usepackage[only,llbracket,rrbracket,llparenthesis,rrparenthesis]{stmaryrd}
\usepackage{accsupp}

in the preamble, one needs \mkern-5.7mu in the fourth argument to \mathchoice for "correct" scaling. To see what goes wrong in that case, it's best to include a line such as discontinuous small symbols in a footnote\footnote{\(\scriptscriptstyle\llbrace \frac{a}{b} \rrbrace\)}. This might help a potential answerer test things out.

Answer 1

With scalerel, you can definitely take care of the \left \right problem. And I think I took care of the tine & scriptsize problem, too. What I did was to save a good copy of the normalsize bbraces in boxes, so that \lxbrace and \rxbrace would place the boxes containing good copies of the bbraces. I then used scalerel to scale these boxes to the desired sizes. So, essentially \lxbrace and \rxbrace replaced your \llbrace and \rrbrace.

The key here is that I am scaling the \normalsize version of the bbraces, whereas what I think happened to your original glyph is that tiny versions of a LaTeX brace are not merely scaled versions of the normalsize brace, but modified (by LaTeX) for legibility. But inyour case, you don't want modifications to your glyph's components as they are shrunk.

Lastly, scalerel is able to emulate the \left \right thingy to scale your brace up to the proper size, whatever that might be. I did use a width-limited scale, to avoid the scaled brace looking too clunky.

\documentclass{article}
\usepackage{amsmath}
\usepackage{scalerel}
\usepackage[only,llbracket,rrbracket,llparenthesis,rrparenthesis]{stmaryrd}
\parskip 1ex
\parindent 0ex
\usepackage{accsupp} % for ensuring the right Unicode codepoint upon pasting

\newcommand*{\llbrace}{%
  \BeginAccSupp{method=hex,unicode,ActualText=2983}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char102}%
    \mathchoice{\mkern-4.05mu}{\mkern-4.05mu}{\mkern-4.3mu}{\mkern-4.8mu}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char106}%
  \EndAccSupp{}%
}
\newcommand*{\rrbrace}{%
  \BeginAccSupp{method=hex,unicode,ActualText=2984}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char106}%
    \mathchoice{\mkern-4.05mu}{\mkern-4.05mu}{\mkern-4.3mu}{\mkern-4.8mu}%
    \textnormal{\usefont{OMS}{lmr}{m}{n}\char103}%
  \EndAccSupp{}%
}

\begin{document}
\newsavebox{\lXbrace}
\savebox{\lXbrace}{$\llbrace$}
\newsavebox{\rXbrace}
\savebox{\rXbrace}{$\rrbrace$}
\def\lxbrace{\scalerel*{\usebox{\lXbrace}}{\llbrace}}
\def\rxbrace{\scalerel*{\usebox{\rXbrace}}{\rrbrace}}

\noindent
\verb|\tiny|: {\tiny\(\llbrace abc \rrbrace\)}
{\tiny\(\lxbrace abc \rxbrace\)} 
 $<-$scale of original brace

  \verb,ORIGINAL:,%
\indent \(\llbrace \left\llbracket\left\{\left[(a) \cdot
bc^{2^3}\right]^4\right\}^5\right\rrbracket \rrbrace\)
\\\verb,FIXED:   ,%
\def\core{ \left\llbracket\left\{\left[(a) \cdot
bc^{2^3}\right]^4\right\}^5\right\rrbracket}%
\indent \(\scalerel[1.5ex]{\lxbrace}{\core}\scalerel*[1.5ex]{\rxbrace}{\core}\)
bbraces scaled, but limited to 1.5ex width

\end{document}

I edited the original image (via zoom) to show how scalerel can take the bbraces and vertically scale them to fit the mathematical object they enclose. The top equation is without scalerel, the bottom with it.

I have added also a zoom on the \tiny size objects, so that one can see the effect of scaling the normalsize glyph rather than letting LaTeX scale both the strut and the brace and then sticking them together. The left version is letting LaTeX scale the glyphs that compose the character, while the right image is letting scalerel package scale the \normalsize character to \tiny proportions.

---

Based on request to be able to provide horizontal compression, to help make glyphs better match comparable counterparts in the literature, I upgraded the scalerel package today (to v1.3), adding the following composite commands

\newcommand\scaleleftright[4][99in]{%
  \if.#2\def\leftarg{\bl@nk}\else\def\leftarg{#2}\fi%
  \if.#4\def\rightarg{\bl@nk}\else\def\rightarg{#4}\fi%
  \scalerel[#1]{\leftarg}{#3}\scalerel*[#1]{\rightarg}{#3}%
}
\newcommand\stretchleftright[4][10000]{%
  \if.#2\def\leftarg{\bl@nk}\else\def\leftarg{#2}\fi%
  \if.#4\def\rightarg{\bl@nk}\else\def\rightarg{#4}\fi%
  \stretchrel[#1]{\leftarg}{#3}\stretchrel*[#1]{\rightarg}{#3}%
}

\newcommand\hstretch[2]{\stretchrel*{\scalebox{#1}{$#2$}}{#2}}

\newcommand\vstretch[2]{\stretchrel*{#2}{\scalebox{#1}{$#2$}}}

The ...leftright commands provide automated scaling for symbols in the manner of mathmode's \left and \right commands. But the \hstretch and \vstretch commands allow for the glyph stretching that was desired by the user. Using his prior example, I further refined the definition of his white-curly-braces symbol as

\def\lxbrace{%
   \hstretch{0.6}{\scalerel*{\usebox{\lXbrace}}{\llbrace}}}
\def\rxbrace{%
   \hstretch{0.6}{\scalerel*{\usebox{\rXbrace}}{\rrbrace}}}

which provided a 40% horizontal compression of his composite-glyph character. Thus, to compare (now in normal, not tiny size), my original correction to this latest correction:

Answer 2

A poor man's approach. It probably doesn't fit the bill:

\documentclass{article}

\begin{document}

\Huge

$\left\lbrace\mkern-2mu\middle| ABC^{\left\lbrace\mkern-2mu\middle| XYZ\middle|\mkern-2mu\right\rbrace} \middle|\mkern-2mu\right\rbrace$

\end{document}