7

In many of the questions asked here, people recommend the use of control versioning for the write-up LaTeX documents, especially for documents that are large, such as theses.

The issue with doing that is that in order for the traditional control versioning diff to be of any use, sentences must be broken into lines. Otherwise, diff doesn't really provide any insight as to what changed in the document and things get messy really fast.

The plan is for all my new documents to follow the "each-sentence-in-a-new-line" format. But I would like to start using control versioning for some of my existing (and large) documents.

I was wondering whether there's a tool that takes a .tex file as input and spits out a new .tex file in which the sentences are separated by a newline character. I am mainly interested in a UNIX tool, but the more portable, the better.

Note: The problem is not as simple as inserting a newline after every period unfortunately. For example, when the tool finds "e.g." in the text, it needs to be smart enough to avoid inserting a newline there. Or, more of an annoyance instead of a big issue, since the last character in a paragraph is very likely to be a period, it needs to avoid inserting an extra newline there. Maybe a tool that utilises the LaTeX internals to identify when a sentence has actually ended?

Joseph Wright
  • 259,911
  • 34
  • 706
  • 1,036
sudosensei
  • 4,072
  • I suppose Emacs can do that. – giordano Dec 19 '13 at 17:58
  • @giordano I assume it does that with regular expressions? I'd be willing to use Emacas just for the sake of converting my source. Then back to Sublime Text it is. :-) Can you show an example of that please? – sudosensei Dec 19 '13 at 18:03
  • 2
    Since the original conversion happens just once, consider emacs query-replace, turning periods followed by blanks to periods followed by carriage returns. That might handle enough of the problem to make diff useful. – Ethan Bolker Dec 19 '13 at 18:09
  • @EthanBolker Yes, that is simple enough. I can do that myself. But I was just wondering whether there's a more elegant solution. There would be too many things that require fixing otherwise. – sudosensei Dec 19 '13 at 18:12
  • Having separated sentences also helps sync with the document viewer: http://tex.stackexchange.com/questions/121615/texmaker-easy-approach-to-edit-large-paragraphs-jump-to-line-inverse-searc – juliohm Dec 19 '13 at 18:21
  • @sudosensei possible idea: you can try to bind the space key to a lambda that inserts a new line char if the previous char is a period, a space char otherwise. This doesn't convert the text but allows you to separate the sentences as you write. – giordano Dec 19 '13 at 18:48
  • Found this on StackOverflow: convert paragraph into sentence using Perl. – Mike Renfro Dec 19 '13 at 19:32
  • A regular expression would look something like this: /([.,;:!?"”–—])\w+([:uppercase:])/$1 \n$2/, i.e. look for punctuation followed by at least one whitespace (\w or [ \t]) followed by an uppercase letter ([A-Z] would work for English), and replace the whitespace by a space followed by a single newline, keep the rest as is. Note that some RX flavors use \1 instead of $1. – Crissov Dec 19 '13 at 21:47
  • Note that \w is a 'word' character in sed. Something like sed '/\begin{document}/,/\end{document}/s/^([^%]*. )([A-Z])/\1 %\n\2/g' should catch many instances. Note that this is largely untested since you don't provide a MWE to test with. It certainly will not catch sentences beginning with a quotation mark, for example, and will only work reliably for English. ([:uppercase:] as suggested above would remove the latter limitation.) It will also miss cases where the full stop is within a command or where the next sentence starts with a command. It will avoid splitting comments, though. – cfr Dec 19 '13 at 22:20
  • I wasn't aware there was already an answer to this question (despite the fact that the question was different). Apologies. Thanks for the insightful comments, everyone. – sudosensei Dec 26 '13 at 10:10

1 Answers1

6

Convert an old document

Here is my take in Emacs Lisp (Emacs runs on several operating systems, so I think it's really portable). It isn't bullet-proof but can be improved:

(defun mg-split-paragraphs-into-sentences ()
  "Split paragraphs into sentences the current buffer.
Newlines within a sentence are removed."
  (interactive)
  (let ((sentence-end-double-space nil))
    ;; Go to beginning of the buffer.
    (goto-char (point-min))
    (while (null (eobp))
      ;; Move to the end of the next sentence.
      (forward-sentence)
      ;; Skip the white spaces.
      (skip-chars-forward " \t")
      ;; Delete all new lines in the current sentence.
      (save-excursion
    (save-match-data
      (while (re-search-backward
          "\\(\n\\|\r\\)"
          (save-excursion (backward-sentence) (point))
          t)
        (replace-match "" nil nil))))
      ;; Insert a new line if needed.
      (if (null (eq (following-char) ?\n))
      (LaTeX-newline)))))

Emacs assumes by default double sentence spacing (which is useful to distinguish between the end of a sentence and of an abbreviation), but can be told to use single sentence spacing by setting sentence-end-double-space to nil. If your documents use double sentence spacing change the

(sentence-end-double-space nil)

into

(sentence-end-double-space t)

The functions uses LaTeX-newline which is a LaTeX comments aware functions to insert newlines provided by AUCTeX. If you don't use AUCTeX replace

(LaTeX-newline)

with

(newline)

but newline isn't LaTeX comments aware. You can run the function with M-x mg-split-paragraphs-into-sentences RET.

Automatically insert a newline after a sentence while writing new documents

As I suggested in the comments, for new documents one can bind the space key to a function which check whether the sentence is ended.

(eval-after-load "latex"
  '(progn
     (define-key LaTeX-mode-map " "
       (lambda ()
     (interactive)
     (if
         (save-match-data
           (save-excursion
         (re-search-backward (concat (sentence-end) "\\=")
                     (save-excursion (beginning-of-line)) t)))
         (LaTeX-newline)
       (insert " "))))))

If one needs to insert a space after a period can use C-q SPC.

giordano
  • 8,486
  • This looks great. I might actually have to give Emacs a shot. :-) I'll mark this answer as accepted since it is a duplicate question. Thanks for the help. – sudosensei Dec 26 '13 at 10:11