How to create a Graph of Mathematica Commands of the installed version

Question

I always wonder how many inbuilt functions does Mathematica have (of course you can google for it) and how they are connected with each other! So I tried this (v10.1).

SetDirectory[$InstallationDirectory<>"/Documentation/English/System/ReferencePages/Symbols"]
comms = FileNames[];
ncomms = Length[comms]

4613

I believe this is the total number of documented functions. Now the second part where I am stuck in. How to find the connections among them?

What I am thinking is to get the list of functions from the See Also section of an example. Here I do one manually. Let's say I start with Plot. See Also in Plot.nb contains ({DiscretePlot, ListLinePlot, ParametricPlot, PolarPlot, Plot3D, ContourPlot, Graphics, Show}). So I store them.

link["Plot"] = {"DiscretePlot", "ListLinePlot", "ParametricPlot" , 
                "PolarPlot", "Plot3D", "ContourPlot", "Graphics", "Show"};

Now I want to scan each element of the list. For example, I take Plot3D and Graphics

link["Plot3D"] = {"ListPlot3D", "ContourPlot", "DensityPlot", "ParametricPlot3D",
                  "Graphics3D", "ListSurfacePlot3D", "Plot", "Show"};

link["Graphics"] = {"Plot", "ListPlot", "ListLinePlot", "ParametricPlot", "DensityPlot",
                    "ArrayPlot", "RegionPlot", "ContourPlot", "Show", "Graphics3D",
                    "Image", "Import", "Sound"};

Now combine and map them

map = Join[DirectedEdge[num["Plot"], num[#]] & /@ link["Plot"],
           DirectedEdge[num["Plot3D"], num[#]] & /@ link["Plot3D"],
           DirectedEdge[num["Graphics"], num[#]] & /@ link["Graphics"]];

Graph[map, VertexLabels -> Table[n -> sets[[n]], {n, nsets}], ImagePadding -> 10]

How can I put this whole thing in a Mathematica code?

Answer 1

(* docs on my system (10.3, Windows) *)
base = FileNameJoin[{$InstallationDirectory, "Documentation",
    "English", "System", "ReferencePages", "Symbols"}];

FileNames[FileNameJoin[{base, "*"}]] // Length

4811

(* symbols in See Also section *)
also[name_String] :=
 Module[{import},
  import = Import[
    FileNameJoin[{base, name <> ".nb"}], {"Cells", "SeeAlso"}];
  Cases[import, TextData[s_String] :> s, Infinity]]

also["I"]

{"Complex", "Re", "Im", "ComplexExpand", "GaussianIntegers"}

edges[start_String, n_Integer] :=
 Reap[Module[{visited = {}},
     Nest[
      Function[visiting,
       Module[{temp},
        temp = Join @@ Table[
           Sow[DirectedEdge[v, #] & /@ also[v]], {v, visiting}];
        visited = Join[visited, visiting];
        Complement[Last /@ temp, visited]]],
      {start}, n]]][[2, 1]] // Flatten

graph[edges : {__DirectedEdge}, start_String, opts___] :=
 Graph[edges, opts,
  VertexLabels -> "Name",
  VertexStyle -> {start -> Red}]

Example 1

With[{x = "I"}, graph[edges[x, 2], x]]

Example 2

(* this took me 40 seconds *)
e5 = edges["Plot", 5];

g = graph[e5, "Plot",
  VertexLabels -> None,
  EdgeStyle -> Directive[Opacity[.2]],
  EdgeShapeFunction -> ({Arrowheads[.01], Arrow[#]} &)];

Through[{VertexCount, EdgeCount}@g]

{1463, 4192}

Majority of vertices are not reachable from a general source (dark red area). Note that: a) graph is directed, and b) crawler stopped after five steps.

dm = GraphDistanceMatrix[g];
dm // MatrixPlot

max = Max[dm /. Infinity -> 0]

13

pos = Position[dm, max];
pairs = VertexList[g][[#]] & /@ pos;

Example of a longest shortest path (via pairs). Starting in Help with Repeated, 13 clicks are needed at least to get to Arrow.

sp = FindShortestPath[g, "Repeated", "Arrow"];
(* {"Repeated", "BlankSequence", ... "Arrow"} *)

HighlightGraph[g, PathGraph[sp, DirectedEdges -> True]]

Answer 2

There is a way for you to find out the relations without internet, and I think it fit your need more as they are actually extracted directly from those "See Also" links.

I'm currently using v11, 5000+ Functions, it takes about 7 min before it finishes the main evaluation, and another 10 min or so for plotting the community graph out, but it's totally internet free, thus can be a bit more helpful when under bad net conditions.

dir = FileNameJoin[{$InstallationDirectory, 
    "Documentation\\English\\System\\ReferencePages\\Symbols"}];
files = Import[dir];

rules = Flatten[Function[{name}, Thread[StringSplit[name, "."][[1]] -> 
   Block[{f = Import[FileNameJoin[{dir, name}]], p}, 
    p = Position[f, "See Also"]; 
     If[p != {}, Cases[#, TextData[s_String]:>s] &@
      Extract[f, Drop[p[[-1]], -5]], {}]]]] /@ files];

Graph@rules
CommunityGraphPlot@%

This method follows how we humans know about the links between functions-----via checking the "See Also" part of the documentation of course. So we can extract those part out one by one, make rules out of them, then use Graph and CommunityGraphPlot to visualize them.

The result is stylish(If we view in the Abstract Painting's style :P) and the structure is clear(Though a bit messy): Mathematica's core functions are closely related!!!

Answer 3

Something like this:

g = SimpleGraph@Graph[
    Catenate[
     Thread /@ 
      Normal@DeleteMissing@
        WolframLanguageData[WolframLanguageData[], "RelatedSymbols", "EntityAssociation"]
     ],
    DirectedEdges -> False
    ];

It's slow, like most new *Data functions, but it works.