pandoc codeblock containing Unicode characters

Question

Motivation

In Julia, many Unicode characters are allowed as identifier. This facilitates the reading of algorithms containing greek letters like α, λ, ετc.

Question

How to fix the following Markdown codeblock so that pandoc can compile it to PDF?

```julia
 = 2
```

Minimal example test.md.

Error received

$ pandoc --pdf-engine=xelatex test.md -o test.pdf
[WARNING] Missing character: There is no  in font [lmmono10-regular]:!

I tried

---
mainfont: Liberation Serif
---
 = 2

but I'm getting the same error.

(Edited)

My TeX Live 2020 :

I've tried mixing normal text with emojis and compile with LuaLaTeX, but only emojis were shown when the main font is set to Noto Emoji Color.

(Edit) Thanks to comment, I've managed to get LuaLaTeX working with

\documentclass{minimal}
\usepackage{fontspec}
\directlua
{luaotfload.add_fallback
 ("myfallback",
  {
   "NotoColorEmoji:mode=harf;"
  }
 )
}
\setmainfont{LatinModernRoman}[
 RawFeature={fallback=myfallback}
]
\begin{document}
It  lorem  \LaTeX.
\end{document}

My new try:

---
header-includes: |
  \directlua
  {luaotfload.add_fallback
   ("myfallback",
    {
     "NotoColorEmoji:mode=harf;"
    }
   )
  }
mainfont: LatinModernRoman
mainfontoptions:
- RawFeature={fallback=myfallback}
---
It  lorem  \LaTeX.

Error received:

$ pandoc --pdf-engine=lualatex test.md -o test.pdf
Error producing PDF.
! Undefined control sequence.
<argument> \LaTeX3 error: 
                Misplaced equals sign in key-value input on line 17
l.17 \fi

As it says you have to use a mono font that contains these chars. And no, I do not know any. — daleif, Oct 23 '20 at 13:40
@daleif Thanks for response. The font Noto Color Emoji is installed by default on my Ubuntu 20.04. I tried setting mainfont: Noto Color Emoji but failed. I wonder what can be done. I have two ideas : 1. try lualatex with some special pakcages 2. try last resort. — GNUSupporter 8964民主女神地下教會, Oct 23 '20 at 16:13
@UlrikeFischer Thx for response. I tried replacing xelatex with lualatex for --pdf-engine, but I ran into the problem of "missing loca table". lualatex-dev has to be used, which is not supported by pandoc. — GNUSupporter 8964民主女神地下教會, Oct 23 '20 at 23:21
@UlrikeFischer Thanks again for your comment. However, typing pandoc --pdf-engine=lualatex test.md -o test.pdf gives me an error of "missing loca table" as mentioned in my previous comment. I wonder if there's a way to get this simple codeblock compiled by pandoc. — GNUSupporter 8964民主女神地下教會, Oct 24 '20 at 11:17
@UlrikeFischer I think so. I installed texlive 2019 directly from the TUG, and I recently upgraded it to 2020. The MWE (min working e.g.) in the linked TeX.SE question worked for me, so I think my TeX Live installation shld be fine. From the linked GitHub issue in a dupe question, the developers of the font think the "missing loca table" behavior is expected. I wonder if we can translate \setmainfont{Noto Color Emoji}[Renderer=Harfbuzz] from TeX to pandoc, especially the part in []. — GNUSupporter 8964民主女神地下教會, Oct 24 '20 at 14:26
don't guess. Look in a log-file. It should tell which version you use. I have no idea if pandoc allows to set font options, but you probably can enable harfbuzz as a default font feature. — Ulrike Fischer, Oct 24 '20 at 14:42
@UlrikeFischer I've included a screenshot of the Terminal showing TeX Live 2020. This question contains detailed info about changing font in pandoc yaml header. I just dunno how to translate harfbuzz for pandoc. — GNUSupporter 8964民主女神地下教會, Oct 25 '20 at 20:41
well did you try to add Renderer=Harfbuzz to the mainfontoptions options section? — Ulrike Fischer, Oct 25 '20 at 20:48
@UlrikeFischer Thanks now I get only , and nothing more. I've given the pandoc user manual a more careful read. The *fontoptions allows options in fontspec, so my question becomes a LuaLaTeX question about compiling text mixed with emoji. — GNUSupporter 8964民主女神地下教會, Oct 25 '20 at 21:08
as you are using lualatex you can define a fallback, see the luaotfload documentation or https://tex.stackexchange.com/a/565896/2388 — Ulrike Fischer, Oct 25 '20 at 21:12
@UlrikeFischer Thanks for your links. The line "NotoColorEmoji:mode=harf;" in direct Lua gives the desired result in LuaLaTeX. However, I ran into a problem with mainfontoptions: - RawFeature={fallback=myfallback} when I try to compile it with pandoc. LaTeX 3 is complaining abt the "Misplaced equals sign in key-value input on line 17" l.17 \fi. — GNUSupporter 8964民主女神地下教會, Oct 25 '20 at 22:28

David Purton · Accepted Answer · 2020-10-26T02:14:05.787

You are getting the errors because pandoc treats the header as Markdown. You need to tell it to treat your header as a LaTeX code block.

However even after doing this, it still fails because the template puts the font selection before it inserts the header material. So you need to put the font selection in your header for things to happen in the right order.

You can use:

pandoc -s test.md -o test.tex

to see the intermediate tex file and diagnose what is going wrong.

Putting it all together, this will work:

---
header-includes:
- |
  ```{=latex}
  \directlua{luaotfload.add_fallback(
               "myfallback",
               {"NotoColorEmoji:mode=harf;"}
             )}
  \setmainfont[RawFeature={fallback=myfallback}]{LatinModernRoman}
  \setmonofont[RawFeature={fallback=myfallback}]{LatinModernMono}
  ```
---
It  lorem  \LaTeX.
 = 2

And then run:

pandoc --pdf-engine=lualatex test.md -o test.pdf

which gives:

Thanks you for your answer. positve one and accepted. I appreciate the first part (problem investigation) and the MWE. — GNUSupporter 8964民主女神地下教會, Oct 26 '20 at 08:20

pandoc codeblock containing Unicode characters

Motivation

Question

Error received

1 Answers1