11

I've been trying to use the hyphenation patterns embedded in the file dehyphtex.tex, which is part of the hyphen-german package. Unfortunately, the file seems to be encoded in a system that's neither UTF-8 nor ASCII. Hundreds of words listed in the file contain characters (mostly vowels with Umlaute, but also e-with-sharp-accent and others) rendered as �. With MacTeX2012, I use TeXworks as my editor; this editor uses UTF8 as the default input encoding scheme. So far, the trial-and-error method of reloading the file using any of the several dozen alternative input encoding schemes that this editor is familiar with has produced no success.

I guess I could hand-edit the file to replace all � instances with the correct UTF8-encoded characters, but I'm hoping there's a more automated way of doing this. Does anyone know off-hand which input encoding scheme is used for this file, and/or does anyone know of a handy method to convert a file of unknown input encoding into a UTF8-encoded file?

lockstep
  • 250,273
Mico
  • 506,678
  • 3
    it is a latin1 encoding , same as ISO 8859-15. With command recode you can change it –  Dec 17 '12 at 21:07
  • Open it with Emacs, go to the end of the file, replace latin-1 with utf-8 and save (a copy of) the file. :) – egreg Dec 17 '12 at 21:18
  • @Herbert - thanks! I didn't know about the correspondence between latin-1 and ISO-8859-15... – Mico Dec 17 '12 at 21:30
  • @Herbert, Just to be precise, latin1 is ISO-8859-1, while ISO-8859-15 is called latin9. Confusing, yes. Concerning accented letters both encodings are identical, the diffrences being in some infrequent characters and mostly in the encoding of the euro symbol, which was not present in latin1 and later added in latin9 – JLDiaz Dec 18 '12 at 00:29
  • 3
    On a Unix system the iconv program does the conversion. A freeware program running also on Windows is Charco – egreg Dec 18 '12 at 00:32
  • 6
    In windows there is also Notepad++, a nice editor, which can read and save any encoding. – JLDiaz Dec 18 '12 at 00:36
  • @Herbert - Please consider posting your comment as an answer; I'll gladly upvote & accept it. – Mico Dec 18 '12 at 11:34
  • 1
    @Mico: done and added an example code line –  Dec 18 '12 at 11:44

1 Answers1

10

it is a latin1 or latin9 encoding , same as ISO 8859-1 or 15. With command recode or iconv you can change it:

recode --diacritics --touch --verbose latin1..UTF-8 <file>

or

iconv -f LATIN1 -t UTF-8 inputfile.tex > outputfile.tex
Mico
  • 506,678
  • Just for completeness (since I mentioned using this editor in my question): In the TeXworks editor, one can proceed as follows to change the file's encoding: (i) open the file in TeXworks; (ii) locate a box in the ribbon along the bottom of the editor window that (likely) says UTF-8; (iii) click on that box and select, e.g., ISO-8859-15 from the menu that presents itself; (iv) click on the same box (which will now say "ISO-8859-15") and select the new menu choice, at the very top, that says Reload using selected encoding. Done! – Mico Dec 18 '12 at 17:07
  • 2
    @Mico: that is possible for a lot of other editors, but really dangerous if you mismatch the encodings ... –  Dec 18 '12 at 17:31
  • Indeed, if the old encoding isn't selected correctly, bad outcomes will likely follow. – Mico Dec 18 '12 at 17:32