How can I reformat a table? Using Markdown > Pandoc > PDF

Question

I am using Markdown to write a document. It's in lots of MD files which are then combined into 1, then sent to Pandoc as follows:

pandoc -s combined.markdown --from markdown+table_captions+auto_identifiers --filter mermaid-filter.cmd --pdf-engine=xelatex -o output.pdf

This works fine, but the table format in the PDF looks like this:

I would very much like to have the table look like this:

Or even better, I'd like to be able to do this:

A simplified combined markdown file is here:

---
author:
- Matthew Petty
affiliation: None
date: \today
title: Title of Document
subtitle: Subtitle of Document
header-includes:
    - '\newcommand{\projectNumberCode}{CODE }'
    - '\newcommand{\projectName}{Project Name }'
    - '\newcommand{\coreSystemName}{Core Name }'
    - '\newcommand{\bt}[1]{\fcolorbox{gray}{lightgray}{#1}}'
    - '\defaultfontfeatures{Extension = .otf}'
    - '\usepackage{fontawesome}'
    - '\usepackage{tocloft}'
    - '\usepackage{graphicx}'
    - '\usepackage{hyperref}'
    - '\usepackage{float}'
    - '\usepackage{glossaries}'
    - '\setglossarystyle{altlistgroup}'
    - '\usepackage{xparse}'
    - '\usepackage{lscape}'
    - '\makenoidxglossaries'
documentclass: article
fontsize: 10pt
secnumdepth: 4
classoptions:
    - a4paper
    - portrait
mainfont: Arial.ttf
geometry:
- top=2cm
- left=1cm
- right=1cm
- bottom=2cm
linkcolor: Blue
numbersections: true
---
\pagebreak
\tableofcontents
\setcounter{table}{0}
\listoftables
\pagebreak
Introduction
Purpose
This document is testing how to format tables in Markdown/Latex
Table: Test Table



  
Heading 1
  
Heading 2
  
Heading 3




  
This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1.
  
This is Contents 2
  
This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3.


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3



\begin{center}END OF DOCUMENT\end{center}
\hrulefill

I also got Pandoc to create a TEX file, which is here: https://pastebin.com/wFArVtuh

I don't know where this would be set - in Pandoc or Latex? Can anyone help?

Answer 1

Pandoc does not support vertical lines in tables, see https://github.com/jgm/pandoc/issues/922 for some discussion.

A workaround is to modify the implementation of longtable from the Pandoc preamble to always print vertical lines. For this the column specification @{}lll@{} needs to be changed into @{}|l|l|l|@{} through Pandoc.

This can be done with the following steps:

1. remove @{} from the start of the specification using \StrGobbleLeft from the xstring package. Note that this counts as two characters, not three.
2. remove @{} from the end using \StrGobbleRight.
3. substitute all occurrences of l with |l using \StrSubstitute.
4. add | at the end and surround by @{} left and right again.

The code that handles the column specification in longtable is a call to \@mkpream (short for make preamble) with argument #2. You can use the functions from the xstring package to change the argument, store it in a new macro, and then call \@mkpream with this new macro, as follows:

\StrGobbleLeft{#2}{2}[\pream]
\StrGobbleRight{\pream}{2}[\pream]
\StrSubstitute{\pream}{l}{|l}[\pream]
\@mkpream{@{}\pream|@{}}

This code can be added to the longtable macro \LT@array using \patchcmd from the etoolbox package. This gives you vertical lines for the full table. Note that this does not work if you use the array package (then some more modifications are needed). By extension it also doesn't work if you use packages that load array internally, such as colortbl.

The substitution now works only on left-aligned columns, you can add similar substitutions for center and right aligned columns.

Pandoc uses the booktabs package with \toprule, \midrule and \endrule instead of \hline. Booktabs introduces a bit of extra vertical space, so when using vertical lines in the table they leave a gap, see Reducing the gap around the frames of longtable. That question also provides a solution, which is to set \aboverulesep and \belowrulesep to 0. This should be added to the Pandoc preamble.

Then a horizontal line can be added after every row by redefining \LT@tabularcr (the longtable macro for the end of a table row) to include a \hline. For this to work also \midrule should be redefined to do nothing, as that line is now already printed by \LT@tabularcr. You can also consider redefining \toprule and \bottomrule, but if you don't change those you get a thick line effect from the combination of \toprule/\bottomrule and \hline.

If you want the vertical margins of the cells to be a bit larger you can add for example \renewcommand{\arraystretch}{1.3}.

Note that these kinds of modifications disable the normal use of longtable and booktabs, so they should not be used when writing a normal LaTeX document. However, for Pandoc it might be ok, because the LaTeX code produced by Pandoc has a restricted predefined format where the effect of modifications is more predictable.

MWE

---
author:
- Matthew Petty
affiliation: None
date: \today
title: Title of Document
subtitle: Subtitle of Document
header-includes:
    - '\newcommand{\projectNumberCode}{CODE }'
    - '\newcommand{\projectName}{Project Name }'
    - '\newcommand{\coreSystemName}{Core Name }'
    - '\newcommand{\bt}[1]{\fcolorbox{gray}{lightgray}{#1}}'
    - '\defaultfontfeatures{Extension = .otf}'
    - '\usepackage{fontawesome}'
    - '\usepackage{tocloft}'
    - '\usepackage{graphicx}'
    - '\usepackage{hyperref}'
    - '\usepackage{float}'
    - '\usepackage{glossaries}'
    - '\setglossarystyle{altlistgroup}'
    - '\usepackage{xparse}'
    - '\usepackage{lscape}'
    - '\makenoidxglossaries'
    - '\usepackage{etoolbox}'
    - '\usepackage{xstring}'
    - '\setlength{\aboverulesep}{0pt}'
    - '\setlength{\belowrulesep}{0pt}'
    - '\renewcommand{\arraystretch}{1.3}'
    - '\makeatletter'
    - '\patchcmd{\LT@array}{\@mkpream{#2}}{\StrGobbleLeft{#2}{2}[\pream]\StrGobbleRight{\pream}{2}[\pream]\StrSubstitute{\pream}{l}{|l}[\pream]\@mkpream{@{}\pream|@{}}}{}{}'
    - '\def\midrule{}'
    - '\apptocmd{\LT@tabularcr}{\hline}{}{}'
    - '\makeatother'
documentclass: article
fontsize: 10pt
secnumdepth: 4
classoptions:
    - a4paper
    - portrait
geometry:
- top=2cm
- left=1cm
- right=1cm
- bottom=2cm
linkcolor: Blue
numbersections: true
---
\pagebreak
\tableofcontents
\setcounter{table}{0}
\listoftables
\pagebreak
Introduction
Purpose
This document is testing how to format tables in Markdown/Latex
Table: Test Table



  
Heading 1
  
Heading 2
  
Heading 3




  
This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1. This is Contents 1.
  
This is Contents 2
  
This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3. This is Contents 3.


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3


  
This is Contents 1
  
This is Contents 2
  
This is Contents 3



\begin{center}END OF DOCUMENT\end{center}
\hrulefill

Result:

Answer 2

Assuming that you are using Rmarkdown, you can insert directly the LaTeX code, for instance:

```{=tex}
\begin{tabular}{c}
Foo \\
\end{tabular}
```

Another possibility is to make a custom table with the help of R packages and/or the help of a LaTeX package, for example, with xtable (R) and tabulary (LaTeX), a MWE in Rmarkdown could be:

---
output:
  pdf_document: default
header-includes: \usepackage{tabulary}
---
Document is testing how to format tables in Markdown/\LaTeX

# To make fake contents in a data frame. 
a <-    "This is long.content. This is long.content.  
         This is long.content. This is long.content."
b <- "This is medium content."
c <- "Some content. "
df <- data.frame(A=c( a,b,c,b,c), B=c(b,c,c,c,c), C=c(b,a,b,c,c))
colnames(df) <- c("Header 1","Header 2","Header 3")

# To make the a jailed table as asked with the data frame.

library(xtable)
options(xtable.comment = FALSE)
print(xtable(df, align="l|L|L|L|"), hline.after =-1:5, 
include.rownames=FALSE,  tabular.environment = "tabulary", 
width = "\\linewidth")

But for heaven's sake :) whatever the method, don't do this kind of tables with vertical and horizontal rules everywhere. You can change most rules by small spaces, making the table not only more elegant, but above all, more readable.