How to remove accents from text?

Question

I would like to know how I can remove accents from a string. For example, how can I transform "string test áéíóú" into "string test aeiou"? I have to normalize some text to make comparisons, and this would be very helpful.

It's worth noting that this is a follow-up question to this one. — Sjoerd C. de Vries, Nov 11 '12 at 19:33

Murta · Accepted Answer · 2015-03-30T21:12:26.407

21

To remove accents from a string I use this function:

removeAccent[string_] := Module[{accentMap,l1,l2},
    l1 = Characters["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"];
    l2 = Characters["SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"];
    accentMap = Thread[l1 -> l2];
    StringReplace[string, accentMap]
]

So, if you apply it as removeAccent["string test áéíóú"]you get: "string test aeiou"

Update

Now in version 10.1 we have the native function: RemoveDiacritics

RemoveDiacritics["string test áéíóú"] you get "string test aeiou"

Timing comparison using the new RepeatedTiming.

RepeatedTiming[removeAccent["string test áéíóú"]]
RepeatedTiming[RemoveDiacritics["string test áéíóú"]]

> 0.000057
> 0.000015

RemoveDiacritics wins!

edited Mar 30 '15 at 21:12

answered Nov 11 '12 at 19:24

Murta

26,275
6
76
166

1

Ð (ð) is not an accented D (d) but the letter eth without any accents. It should not be considered an accented character in this context. – Oleksandr R. Apr 04 '13 at 14:04
Is RemoveDiacritics not yet documented? – Karsten7 Mar 14 '15 at 03:56
@Karsten7. Almost, see here – Murta Mar 14 '15 at 04:14
You mean in the next version 10.1 ?? – SquareOne Mar 14 '15 at 21:56
@SquareOne yes! – Murta Mar 14 '15 at 22:19
@Murta Did you try this ? 116 new functions ... – SquareOne Mar 15 '15 at 18:13
@SquareOne similar. I just used 2015 in the search box. – Murta Mar 15 '15 at 18:37

score 10 · Answer 2 · answered Nov 11 '12 at 19:30

10

Your own method without extraneous reevaluation:

With[{accentMap =
  Characters /@
    Rule["ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ", 
         "SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"] // Thread},

 removeAccent[string_] := StringReplace[string, accentMap]

]

answered Nov 11 '12 at 19:30

Mr.Wizard

271,378
34
587
1,371

2

Would a Dispatch[] help matters here? – J. M.'s missing motivation Nov 12 '12 at 09:28
1

I don't think so. I have made some tests with Dispatch, and it gets slower. StringReplacemust already do something like it. – Murta Nov 12 '12 at 11:38
1

@J.M. I'll have to test it but I believe that because the String operations are handled by a separate library things like Dispatch are not applicable. At least that's what I seem to recall concluding previously. – Mr.Wizard Nov 12 '12 at 16:39

J. M.'s missing motivation · Answer 3 · 2013-04-10T07:59:08.557

Works nicely:

removeAccent[s_String] := Module[{patt = "(Capital)?([A-Z]{1})([A-Z]\\w*)*", del}, 
 del = Select[Characters[s], StringMatchQ[ToString[FullForm[#]],
                                          RegularExpression[".*\\[" <> patt <> "\\].*"]] &];
 StringReplace[s, Thread[del -> Map[First, StringCases[ToString[FullForm[#]] & /@ del, 
               RegularExpression[patt] :> If["$1" === "", ToLowerCase, Identity]["$2"]]]]]]

Test:

removeAccent["string test áéíóú"]
   "string test aeiou"

removeAccent["Çärîñő Ð Štùrm"]
   "Carino Ð Sturm"

How to remove accents from text?

3 Answers3

Linked