Checking if a word exists

Question

Probably the simplest way to check a word if it exists is to use DictionaryWordQ[]

DictionaryWordQ["color"]

True

But it does not recognise British spelling:

DictionaryWordQ["colour"]

False

So my guess is that DictionaryWordQ[] checks words from WordList[]:

Select[WordList[], StringLength[#] > 3 && StringTake[#, 4] == "colo" &]

{cologne,colon,colonel,colonial,colonialism,colonialist,colonic,colonist,colonization,colonize,colonized,colonizer,colonnade,colonnaded,colony,colophon,color,coloration,coloratura,colored,colorfast,colorful,colorimetric,coloring,colorist,colorize,colorless,colorlessness,colors,colossal,colossus,colostomy,colostrum}

No colour. However,

WordData["colour"]

{{colour,Noun,Color},{colour,Noun,Gloss},{colour,Noun,VisualProperty},{colour,Noun,Timber},{colour,Noun,Interest},{colour,Noun,Variety},{colour,Noun,Race},{colour,Noun,Stuff},{colour,Adjective},{colour,Verb,Discolor},{colour,Verb,Alter},{colour,Verb,TouchOn},{colour,Verb,Rationalize},{colour,Verb,Beautify},{colour,Verb,Influence}}

So the word colour is known to Mathematica, but DictionaryWordQ[] does not see it!

Interestingly,

DictionaryWordQ["qt"]

True

Select[WordList[], StringLength[#] > 1 && StringTake[#, 2] == "qt" &]

{}

which gives an impression that DictionaryWordQ[] is looking at a bigger set of words compared to WordData[] (or it is a bug!).

Question:

What is the best way to verify if a word exists?

(Version used: 11.1, 10.4 in Linux)

@BobHanlon But since no language was given by OP, clearly OED loses out: Dictionary[ {All, "Bürgersteig"}] gives {{German, Bürgersteig}}. :) — gwr, Mar 17 '18 at 18:20
What does it mean for a word to exist? Do the words "mimsy" and "borogoves" exist? — mattdm, Mar 18 '18 at 16:29
@mattdm Play Scrabble, to give a practical example, with a bunch of people and you will soon find out, that citing Lewis Carroll or your local community's mumbo-jumbo may not be enough evidence. :) — gwr, Mar 18 '18 at 17:30

score 14 · Accepted Answer · edited Mar 19 '18 at 10:14

Using DictionaryLookup seems to be more reliable, while not perfect:

DictionaryLookup[ {All, "word" }, IgnoreCase -> True ]

It seems to be rather comprehensive across different languages:

DictionaryLookup[ {All, "qt"}, IgnoreCase -> True ]

{{"Hungarian", "Qt"}}

Albeit, Mathematica will not find "qt" as abbreviation for Quantity (see for example Merriam Webster.

DictionaryLookup[{All, "colour"}, IgnoreCase -> True]

{{"BritishEnglish", "colour"}}

Also, it seems to be more comprehensive than WordList[]:

DictionaryLookup[{"BritishEnglish", "colo" ~~ __}] // Select[StringLength@# > 3 &]

{colocynth,cologne,colon,colonel,colonelcy,colonels,colonial,colonialism,colonialist,colonially,colonials,colonic,colonies,colonisable,colonisation,colonisations,colonise,colonised,coloniser,colonisers,colonises,colonising,colonist,colonists,colonnade,colonnaded,colonnades,colons,colony,colophon,colophony,coloration,coloratura,colossal,colossally,colossi,colossus,colossuses,colostomy,colour,colourability,colourable,colourableness,colourably,colouration,colourcast,coloured,coloureds,colourer,colourers,colourfast,colourfastness,colourful,colourfully,colourfulness,colouring,colourings,colourisation,colourisations,colourise,colourises,colourist,colouristic,colourists,colourless,colourlessly,colourlessness,colours}

Length @ %

68

as opposed to 33 from WordList[], as given by OP.

Alternative to DictionaryWordQ for English Words

Options[englishWordExistsQ] = {
   IgnoreCase -> True
};
englishWordExistsQ[ word_String, opts : OptionsPattern[englishWordExistsQ] ] := Module[
  {
    optIgnoreCase = OptionValue[IgnoreCase],
    results
  },
  results = Join[
    DictionaryLookup[ {"BritishEnglish", word}, IgnoreCase -> optIgnoreCase],
    DictionaryLookup[ {"English", word }, IgnoreCase -> optIgnoreCase ]
  ];
  results =!= {}
]

Now:

englishWordExistsQ @ "colour"

True

DictionaryLookup[{All, "excellent" ~~ ___}] – Sumit Mar 18 '18 at 06:33 — Sumit, Mar 18 '18 at 06:33

J. M.'s missing motivation · Answer 2 · 2018-03-22T15:47:05.470

Since gwr already answered the question, let me offer a few routines to check alternative dictionaries.

Merriam-Webster

Merriam-Webster provides an API for looking words up in their Collegiate® Dictionary. You will need to register to obtain an API key:

$MWAPIKey = (* insert your API key *);

MWLookup::suggest = "Word `1` not found. Returning a list of suggestions instead.";
MWLookup::noword = "Word `1` not found.";

MWLookup[word_String] := Module[{url, raw, check},
  url = "https://www.dictionaryapi.com/api/v1/references/collegiate/xml/";
  raw = Import[url <> URLEncode[ToLowerCase[word]] <> "?key=" <> $MWAPIKey, "XML"];
  check = Cases[raw, XMLElement["entry", {"id" -> s_String}, rest_] :> s, ∞];
  If[check =!= {}, check,
     check = Cases[raw, XMLElement["suggestion", {}, {s_String}] :> s, ∞];
     If[check =!= {}, Message[MWLookup::suggest, word]; check, 
        Message[MWLookup::noword, word]; check]]]

For example,

MWLookup["colour"]
   {"colour"}

MWLookup["enfant terrible"]
   {"enfant terrible"}

MWLookup["poiuyt"]

MWLookup::suggest: Word poiuyt not found. Returning a list of suggestions instead.

   {"payout", "pouty", "pout", "Paiute", "Poitou", "peyote", "uppity", "potty", "poult",
    "piety", "pity", "polit", "polity", "polite", "putty", "pilot", "opiate", "poet",
    "puto", "pollute"}

.

MWLookup["qazwrk"]

MWLookup::noword: Word qazwrk not found.

{}

Oxford

The Oxford Dictionaries also provide an API, which you'll need to register for. Their system is a bit more complicated, since calling their API requires an API ID and an API key:

$OxfordAPIID = (* insert your API ID *);
$OxfordAPIKey = (* insert your API key *);

OxfordLookup::noword = "Word `1` not found.";

OxfordLookup[word_String] := Module[{url, w, raw, check},
      url = "https://od-api.oxforddictionaries.com:443/api/v1/inflections/en/";
      w = StringReplace[URLEncode[ToLowerCase[word]], "+" -> "%20"];
      raw = Import[HTTPRequest[url <> w, <|"Method" -> "GET",
                                           "Headers" -> {"app_id" -> $OxfordAPIID,
                                                         "app_key" -> $OxfordAPIKey}|>],
                   "String"];
      check = Quiet[Check[ImportString[raw, "RawJSON"], ImportString[raw, "HTML"],
                          Import::jsonhintposandchar], Import::jsonhintposandchar];
      If[check === "Not Found  \nNo lemmas found matching supplied source_lang and word",
         Message[OxfordLookup::noword, word]; Return[{}, Module]];
         StringReplace[Through[check["results"]["id"]], "_" -> " "]]

For example,

OxfordLookup["colour"]
   {"colour"}

OxfordLookup["enfant terrible"]
   {"enfant terrible"}

OxfordLookup["poiuyt"]

MWLookup::noword: Word poiuyt not found.

{}

Oxford also provides an API for searching the Oxford English Dictionary, but the API is still in prototype stage, and still has some functionality (e.g. stemming) missing. Nevertheless, here is how to call the API from Mathematica:

OEDLookup::noword = "Word `1` not found.";
OEDLookup[word_String] := Module[{url, w, raw}, 
   url = "https://oed-api-demo-2445581300291.apicast.io:443/oed/api/v0.0/words/?lemma=";
   w = URLEncode[ToLowerCase[word]];
   raw = Flatten[Import[HTTPRequest[url <> w, <|"Method" -> "GET", 
                        "Headers" -> {"Accept" -> "application/json", 
                        "app_id" -> $OxfordAPIID, "app_key" -> $OxfordAPIKey}|>], 
                        "RawJSON"]];
   If[raw === {}, Message[OEDLookup::noword, word]; {}, Through[raw["lemma"]]]]

For example:

OEDLookup["set"]
   {"set", "set", "set", "set"}

OEDLookup["to and fro"]
   {"to and fro"}

Very nice. Can you give an estimate for how long an average lookup takes? — Yves Klett, Mar 18 '18 at 11:50
My Internet connection is horribly slow, but I've found that the APIs for both services return in about a second or so. — J. M.'s missing motivation, Mar 18 '18 at 11:53

score 0 · Answer 3 · answered Jan 26 '20 at 23:24

I appreciate Your my very friends. But I have to add some optimization. I suggest thesaurus.com for the task too.

Here is my concept:

request = 
 HTTPRequest[
  "https://thesaurus.altervista.org/thesaurus/v1?word=peace&language=\
en_US&output=json&key=test_only", <|Method -> "GET", 
   "ContentType" -> "application/x-www-form-urlencoded", 
   "UserAgent" -> 
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) \
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 \
Safari/537.36", 
   "Headers" -> <|
     "X-JWToken" -> 
      "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9eyJTZXNzaW9uSUQiOiJtdm5zeGd\
4ZXlicWJ3amZoYXZucHE0ZmEiLCJTaXRlSUQiOiI1NiIsIm5iZiI6MTU1MjQzODUxNCwiZ\
XhwIjoxNTUzMDQzMzE0LCJpYXQiOjE1NTI0Mzg1MTR9.\
3zQMCUdqkLfMTftXZXd9GxuFT8NxOaAFTb1iAHtnNI", 
     "RequestTarget" -> "AJAXService", "DNT" -> "1", 
     "ADRUM" -> "isAjax:true"|>|>]

URLExecute[request]

The result is

The output is

{"response" -> {{"list" -> {"category" -> "(noun)", 
      "synonyms" -> 
       "order (generic term)|war (antonym)"}}, {"list" -> {"category" \
-> "(noun)", 
      "synonyms" -> 
       "harmony (generic term)|concord (generic term)|concordance \
(generic term)"}}, {"list" -> {"category" -> "(noun)", 
      "synonyms" -> 
       "peacefulness|peace of \
mind|repose|serenity|heartsease|ataraxis|tranquillity (generic \
term)|tranquility (generic term)|quietness (generic term)|quietude \
(generic term)"}}, {"list" -> {"category" -> "(noun)", 
      "synonyms" -> 
       "public security|security (generic term)"}}, {"list" -> \
{"category" -> "(noun)", 
      "synonyms" -> 
       "peace treaty|pacification|treaty (generic term)|pact (generic \
term)|accord (generic term)"}}}}

That is indeed already a Mathematica type and Part can operate on it to work with the information given. The advantages are immense versus the other solution given here. This are typed linguistic terms: verb, noun, synonym, antonym, category. The classification can even go further. I have to apologize for this in not the level of the html page from thesaurus.com. Chances are good that some Mathematica coding rises the level appropriate.

So have fun. No limits are at present on the HTTPRequest and the connection, processing and download of collections is fast.

This type of request can be abstracted. It is based on Javascript, the keyword peace and can be turned into a function with string operation. The key is free on

API key for thesaurus.com

for everybody even for commercial purposes and authoring. This can be used for parsing and tagging on texts as well. An example is on the thesaurus.com page for flexibility in better writing. The are several language beside american english. The languages are: English Spanish French German Italian Portuguese.

It can be used as a dictionary as in the other example here too. The is feed with wiktionary.com.

Checking if a word exists

3 Answers3

Merriam-Webster

Oxford

Linked