Post-filtering output that I can't seem to pre-filter with silence package

Question

I have a book that was producing 20,000 lines of output to stdout when I compiled it with pdflatex. I did the following to reduce the output:

\RequirePackage{silence}
\WarningFilter{Fancyhdr}{\headheight is too small}
\WarningFilter{textpos}{environment textblock}
%\pdfsuppresswarningpagegroup=1 % my pdftex isn't new enough
\hfuzz=999pt % reduce overfull hbox errors
\hbadness=10000 % reduce underfull hbox errors

This cut it to 500 lines. However, that is still too much. I would like to cut it to zero lines when there are no errors, so that I can actually look at the output every time I compile the book. The remaining not-useful output seems to consist mainly of the following categories:

1. Messages saying what packages I've loaded: (/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty)
2. A message for every file included via includegraphics.
3. A message for every font used: <</usr/share/texlive/texmf-dist/fonts/type1/p ublic/amsfonts/cm/cmsy10.pfb>>
4. Files being included.

Some of these classes of messages seem to be impossible to prevent, because they're coded into TeX itself. Therefore there seems to be no other option than to do some kind of post-filtering on the output. Do I have to code my own filter, or have other people written something to do this?

Related:

• Use latexmk to filter the log file
• Silence file inclusion messages?
• Can't silence a pdftex "PDF inclusion: multiple PDFs with page group" error
• Reducing the console output of LaTeX

Answer 1

Several of the existing automated build tools have the capability to further reduce the terminal output of a compilation. A relatively new player on the scene, which I will focus on here, is ltx2any. This system offers build output of the form:

$> ltx2any bibtex_test.tex 
[ltx2any] Initialising ...
[ltx2any] pdflatex(1) running ... Done
[ltx2any] bibtex running ... Error
[ltx2any] pdflatex(2) running ... Done
[ltx2any] pdflatex(3) running ... Done
[ltx2any] pdflatex(4) running ... Done
[ltx2any] There were 1 error and 3 warnings.
[ltx2any] Output generated at bibtex_test.pdf
[ltx2any] Assembling log files ... Done
[ltx2any] Log file generated at bibtex_test.log.md

_{Source: ltx2any README.md}

The shortened summary output on the terminal alerts the user of any errors encountered during the build, and more details about any errors may be examined in the Markdown log file (a pdf log file option is also available). The complete, unadulterated log file from (La)TeX is also available in its usual location.

While this doesn't strictly meet your zero-line requirement, the output here would certainly fit on a reasonably-sized terminal window without scrolling, and I think the minimally-invasive visual feedback is nice to have.

The build system ltx2any is highly configurable for many different engines and workflows. Refer to the GitHub repository for more details and examples of its capabilities.

Answer 2

Since nobody seemed to know of a preexisting solution, I wrote the following ruby script.

#!/usr/bin/ruby

# This program filters TeX's output to get rid of stuff I don't care about.
# It reads stdin, writes stdout.

def do_filtering(t)
  path_chars = "[a-zA-Z0-9./\-]" # characters that we expect to see in a path name
  extensions = "(sty|aux|fd|cfg|def|ldf|cls|clo)" # tex tells us about these, e.g., (/foo/bar/baz.sty)
  path = "\.*\/#{path_chars}+(\n#{path_chars}+)?"
  t.gsub!(/\(#{path}\.#{extensions}\)*\s+/,'')

  t.gsub!(/\n*pdfTeX warning: pdflatex \(file #{path}\n?\)\n?:\n? \n?[^\n]+(\n[^\n]+)?included in a single page\n*/,'')
       # http://tex.stackexchange.com/questions/183149/cant-silence-a-pdftex-pdf-inclusion-multiple-pdfs-with-page-group-error

  # messages from includegraphics, e.g.,  <./ch01/figs/reservoir-tangent-line.pdf>
  t.gsub!(/<([^\n]+)\n([^\n]+)>/) {"<"+$1+$2+">"} # remove linebreaks in middle of paths
  graphics_extensions = "(jpg|pdf|png)"
  graphics_file = "#{path}\.#{graphics_extensions}( \(PNG copy\))?"
  t.gsub!(/<(#{graphics_file} )*#{graphics_file}>/,'')

  # Underfull \vbox (badness 10000) has occurred while \output is active
  # Underfull \hbox (badness 10000) in paragraph at lines 126--126
  t.gsub!(/Underfull \\(hbox|vbox) \(badness \d+\) (has occurred while \\output is active|in paragraph at lines \d+--\d+)/,'')

  # Overfull \hbox (6.56241pt too wide) detected at line 35
  # Overfull \hbox (8.53581pt too wide) in paragraph at lines 824--824
  t.gsub!(/Overfull \\(hbox|vbox) \(\d+\.\d+pt too (wide|high)\) in paragraph at (line \d+|lines \d+--\d+)/,'')

  # clean up page numbers like [18] [19]
  t.gsub!(/\[(\d+) +\]/) {"["+$1+"]"} # remove whitespace inside brackets
  1.upto(10) { |i|
    t.gsub!(/(\[\d+\])\n ?(\[\d+\])/) {$1+" "+$2} # remove newlines between page numbers
  }
  t.gsub!(/\[\]/,'') # empty brackets
  t.gsub!(/\$\[\]\$/,'') # empty $[]$
  t.gsub!(/\$\$/,'') # empty $$
  t.gsub!(/^\s*\n/,'') # lines containing only whitespace

  # messages from specific packages, etc.
  date_version = "<[^>]+>" # e.g., <2008/02/07>, used by packages to announce what version they are
  t.sub!(/\AThis is [^\n]+\n/,'')
  t.sub!(/^LaTeX2e #{date_version}Babel[^\n]+languages loaded\.\n/,'')
  t.sub!(/Package: textpos[^\n]+\n/,'')
  # remove lines beginning with specific text
  [
    "For additional information on amsmath",
    "Document Class",
    "Style option",
    "Grid set ",
    "TextBlockOrigin set",
    "ABD: EveryShipout initializing macros",
    "Output written on",
    "Transcript written on",
  ].each { |x|
    t.gsub!(/^#{x}[^\n]*\n/,'')
  }
  t.gsub!(/^ ?\|?\\OT1[^\n]+(application|derivative|intermediate|acceleration)\|? ?$/,'')
  t.gsub!(/^ ?\|?\\OT1[^\n]+/,'') # \OT1/cmr/m/n/10.95 0.110001000000000000000001

  # list of fonts, e.g., <</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmbx10.pfb>>
  t.gsub!(/<\n?<\n?#{path}\n?>\n?>/,'')
  t.gsub!(/{\n?#{path}\n?}/,'')


  t.gsub!(/\n{2,}/,"\n") # blank lines

  return t
end

t = $stdin.gets(nil) # nil means read whole file
if t.nil? then t='' end # gets returns nil at EOF, which means it returns nil if file is empty

do_filtering(t)

print t