49

Introduction

Hello, this is a more general question than is typically asked here. However, I chose to post it here because this seems to be the most active community for Wolfram Language / Mathematica software developers.

I would like to ask what you think needs to be done to the language and tooling to improve adoption among software developers.

Related discussions

There are several other discussions that focus on the woes of deployment, the language's usage in education, etc., but I would like to focus on the barriers to using the language for real development work.

For some background, here are some related discussions I have found:

"Should I learn Mathematica or Python to play with math and statistics stuff?" - https://qr.ae/pGlzma

"Why are some professors negative on Mathematica" - https://community.wolfram.com/groups/-/m/t/413906

Summary

Please let me know if you have any thoughts about features or improvements that would make WL easier for you to use in 'real' software development projects.

p.s. This is to organize my own thoughts as much as it is to get input from others. Thank you for your help.

Alec Graves
  • 1,058
  • 7
  • 13
  • 4
    I feel like this might go better on the meta SE, but this is a good discussion nonetheless. It may, possibly, already be a duplicate as well, but I am not at all confident in either of these points. – CA Trevillian Dec 05 '21 at 22:35
  • 1
    Every decade or so I take a stab at it, but then get frustrated by the syntax (and somehow cannot find a good documentation/tutorial on that syntax): could you link to one? – lalala Dec 06 '21 at 11:39
  • 6
    I don't think of WL as a language competing for developer adoption / market share with python / julia / java / c++ etc. It's better to think of it as a research tool / environment. Most start their Mathematica journey doing integrals, factoring polynomials, and drawing plots, not learning the syntax - most come for the massive pool of built in features. – flinty Dec 06 '21 at 13:01
  • 3
    The need to get used in more papers - particularly the more visual ones like you see in call-for-papers at SIGGRAPH, so support for a render context and shaders would help. A lot of real world problems have geometry, so a more rich + robust CSG system and scene graph would be nice. Obviously parallel / GPU programming could be better, esp useful for things like big point clouds. – flinty Dec 06 '21 at 13:16
  • (side note: there are some free (online cloud) interpreters for Mathematica – TIO (text-only), Wolframcloud (requires creating an account), etc.) – user202729 Dec 07 '21 at 08:45
  • 4
    A good start would be fixing the numerous bugs in the Linux GUI, which have been present for over 5 years when I first started using Mathematica. – Myridium Dec 08 '21 at 05:47

6 Answers6

38

OP's take

Why I like WL

  • My time is valuable. Learning usage of several new functions from the WL docs takes less time than sifting through the disparate documentation and learning OO class structures for several new Python libraries.
  • WL provides a common interface to many domains: global optimization, GPU-accelerated NN training, audio processing, image processing, NLP, etc.
  • The interface to core language functions is pretty consistent across these many domains.
  • The documentation content is usually good. It has example usage for almost everything.
  • I agree with some of the arguments made here: https://blog.wolfram.com/2019/04/02/why-wolfram-tech-isnt-open-source-a-dozen-reasons/
  • There are some genuinely good ideas incorporated into the ecosystem: the neural network repository, data repositories, etc.
  • The language is actually good: core everything-is-a-symbol nature, pattern matching and rewrite rules, functional programming support, flexible syntax, etc.

Why I am not using WL professionally as a developer ​

  • The Mathematica/'Wolfram Desktop' GUI is bad.
    • The GUI is simply not responsive. The code-complete functionality and documentation window will often freeze if you try to use the GUI while running a computation.
    • Opening and scrolling through documentation pages should be as fast as scrolling through local HTML pages on chrome. Instead, the native documentation viewer commonly lags and freezes when scrolling through pages that should consist primarily of static text and images.
    • 'Aborting' the current computation will frequently not work and require manually killing kernels. If I try to interrupt a computation, it should stop immediately.
    • Sharing and presenting data in wolfram notebooks is just not as convenient as it is when using Jupyter notebooks. Even with cloud publishing, the web interface is simply slow.
    • Naturally, I will compare the GUI's performance to that of Jupyter notebooks. If web browsers are able to more quickly render some text boxes and provide basic code completion than a dedicated GUI, something needs to be rewritten.
  • There is very little developer tooling and support
    • 'real' WL developer tools are tucked into an Eclipse plugin. Who still uses Eclipse?
    • The most commonly used IDEs and editors (JetBrains, VSCode, Sublime) lack good language support (correct syntax checking, debugging, autocomplete, documentation rendering, etc.) for WL
    • This kind of tool integration should be a top priority if WRI wants WL to be used in real projects by devs.

^ Edit: IDE/editor support is getting better with Mr. Brenton Bostick's effort on plugins for VSCode and Sublime using Microsoft's Language Server Protocol. I would still like to see better-maintained Intellij integration (perhaps using the language server like the Sublime and VSCode plugins).

  • There are artificial restrictions on the number of active processes and 'wolfram kernels' you can use.
    • This one really puts a bad taste in my mouth. If I was going to pay hundreds/thousands of dollars to use a programming language with a giant standard library, I should be able to write and execute as many programs as I want at a time on my machine. Limiting the number of active WL processes to two is a strange decision in the personal-computer era.
    • When I build a new computer, it will have a minimum of 16 cores and 32 threads. This is the direction computing hardware is going. I should be able to spin up as many 'wolfram kernels' as can fit in my system's memory.

Some things that have gotten better recently

  • Deployment

    • Between 'Wolfram Engine', docker containers, and CloudDeploy, the outlook is far less bleak on this front than it was in the past.
    • Also, several GB of data is not as bad as it was in the past for many users. Many common video games can be hundreds of GB these days.

  • General execution speed (when running through wolframscript, far away from the GUI)

    • With code compilation, efficient matrix math operations, and GPU-accelerated computation via NeuralNetwork functionality, the overall language execution speed is not terrible.
    • Also, computers have continued to get faster with bigger GPUs, CPU cache size increases, greater memory bandwidth, and SSD proliferation.

  • Tooling

    • As mentioned above, there are now official plugins for VSCode and Sublime with propper parsing and syntax highlighting. Autocomplete could still be better. Also, the plugins could use a debugger.
Alec Graves
  • 1,058
  • 7
  • 13
  • 11
    no mention of lack of an easy to use debugger under Why I am not using WL professionally as a developer? under There is very little developer tooling and support. I think this should be on your list. This for me is the most important item missing in WRI development platform. If you take the debugger away from other developers using other languages, there will be riots on the streets. Imagine having to develop a 100,000 lines large application without a debugger. – Nasser Dec 06 '21 at 00:29
  • 3
    Regarding support for VSCode and Sublime, take a look at this presentation from WTC-2021. GitHub repo for Sublime and repo for VSCode. Looks promising. – Rohit Namjoshi Dec 06 '21 at 01:08
  • 3
    The most commonly used IDEs and editors (JetBrains, VSCode, Sublime) lack good language support (correct syntax checking, autocomplete, documentation rendering, etc.) for WL But there is Halirutan award winning Mathematica plugin for JetBrains ? – Nasser Dec 06 '21 at 01:23
  • Halirutan's plugin has the best autocomplete I have used so far (when Halirutan takes the time to update it for new releases of WL ). And thanks Rohit, I did not now about language server implementation and official support for sublime and VSCode plugins! This seems like an exciting development. – Alec Graves Dec 06 '21 at 03:29
  • 4
    I've never understood the 'tooling' thing. The state of tooling outside of MMa is arguably worse: look at C++ where you have no default tools whatsoever. Compilers, debuggers, sanitizers, build systems, IDEs, even libc/libstdc++ are separate things from the language - an unopinionated standard with plenty of undefined behaviour. C/C++/Java etc. these languages suffer from dependency hell, and too much useful stuff living in 3rd-party libraries, creating a big versioning problem. MMa tries to provide more out of the box, while these other languages outsource the hard work to 3rd parties. – flinty Dec 06 '21 at 13:27
  • 5
    ^ and then there's python. Think of how much effort and money is wasted managing docker images, pip installing etc. Python has changed very slowly over the years while modules have popped up everywhere effectively chaining the language down and slowing the transition from 2 to 3. I like to see new useful features in MMa that have been well thought out (yes some things could be more robust), but I'm not sure mass adoption is necessarily something I want if it makes my experience of MMa worse in future and delays evolution of the features and language. – flinty Dec 06 '21 at 13:39
  • 2
    Flinty, while it may be true that there are no ‘default’ tools for C++/Java/Python, I would argue that the time invested by JetBrains, Sublime, and VSCode have allowed these tools to far exceed the Mathematica GUI’s capabilities for standard algorithmic work.

    And now, there is even DataSpell, an IDE integrated tightly with Jupyter, providing solid graphical REPL capabilities similar to Mathematica GUI, but with excellent code-complete. Mathematica has some catching up to do.

    And package development is important. Compare how easy it is to make/distribute Node or Pip modules vs Packlets.

     – Alec Graves Dec 06 '21 at 18:04
  • 1
    @AlecGraves Yes that's my problem with the expectation though - the standards committee doesn't pay for those - 3rd parties with more resources do, for profit or to improve productivity. Jupyter is a non-profit but has sponsors. Realistically while some of the things you mention must be fixed (esp debugger and arbitrary constraints on kernels are bad), it's on the community, including corporate users, to come up with packages and enrich the ecosystem. Wolfram has a few hundred employees and isn't that big (it's not publicly traded either) so it's a chicken-and-egg situation. – flinty Dec 07 '21 at 00:19
  • 2
    Not an answer, as there isn't a solution to this, but my biggest gripe with WL is how complicated the evaluation mechanism is. I understand it was developed with symbolic processing in mind, but it can make it really difficult to reason about code when you have upvalues, downvalue, and numerous steps to an evaluation cycle, not to mention the possibility of repeated re-evaluation. – Michael Dec 07 '21 at 04:06
  • 1
    ad "Who still uses Eclipse?": I do. And many other people who actually care about good, free (as in freedom) software. Sure, there are good commercial alternatives, but many people do not want to support them. – JohnEye Dec 07 '21 at 11:28
  • 9
    "There are artificial restrictions on the number of active processes and 'wolfram kernels' you can use" my +1 for raising this issue. It doesn't make any sense that my desktop has 32 threads and my multi-thousand prog language cannot use them due to a license issue!! – Hans Olo Dec 07 '21 at 15:40
  • 2
    @Nasser Please do not use code formatting for quotations, or for anything that isn’t meant as code (read: intended primarily for computer consumption rather than human consumption). It’s not just a highlight, it’s a semantic tag that means the text is code, and that means alternative browsing technologies, e.g. screen readers for the blind, have to try to interpret it as code. This can make your comments (and thus the page overall) harder to understand for those using such technologies, reducing accessibility. Quotation marks are the standard tool for indicating quotation. – KRyan Dec 07 '21 at 16:42
  • JetBrains: https://plugins.jetbrains.com/plugin/7232-wolfram-language – Jon McLoone Dec 10 '21 at 11:02
23

As an academic in math and engineering, I like many of Mathematica's abilities, especially the ability to quickly prototype algorithms and check analytical results.

In addition to the many points mentioned, there is a major problem from a literature and research perspective: built in commands and solvers are generally black boxes with no references or citations.

Taking a function at (somewhat) random, consider SpherePoints which "gives exactly equally spaced points in certain cases for small n. In other cases, it places points so they are approximately equally spaced." For what n is it exact or approximately equally spaced? How approximately equally spaced are these points, i.e., what tolerance is given? What algorithm is it using to do so and where is this algorithm in the literature? Is WRI's implementation of this algorithm correct for all n? What, exactly, is WRI's definition of "equally spaced" and how does it relate to the Thomson Problem and the Tammes Problem? As such, academic rigor and proper literature references generally require rebuilding built-in functions from scratch, with the built-in as an error-checker or useful tool in the rapid prototyping stage. And this is only a simple built-in command, not to mention more complex solvers and algorithms.

This seems to be a general weakness of WRI and Stephen Wolfram in particular. Consider that ANKOS is a ~1,000 page "research" tome that comes with a big old copyright notice, but fails to include proper citations and references. If such a document were submitted to a college course or as a dissertation/thesis, it would likely involve charges of plagiarism...

Edit: Due to continued strong interest in this question, I wanted to add some more information and examples to my answer.

  • Due to the discussed black box nature, my workflow generally uses Mathematica in the early prototyping stages. However, I then recreate any algorithms in Python/C++/... with commented and referenced code before finalizing a project. In this way I can ensure that I have proper literature references and any mistakes in my implementation are hopefully traceable for future researchers, giving code and data upon which I am willing to stake my academic reputation.

  • When proving analytical results, Mathematica can be a great boon in simplifying complex expressions or computing complex integrals. However, the standards of a mathematical proof require figuring out the necessary steps. This is often easier than trying to do so without Mathematica's help---knowing the end result often makes analytical work significantly easier. This makes me think of an SMBC comic, that "Mathematica said so" does not constitute a valid mathematical proof.

  • Mathematica is quite clever (hence my continued use), but is occasionally too clever for me to predict (especially in conjunction with the black box nature). Consider simple commands 200!, Gamma[201], Product[i, {i, 200}], and Fold[Times, Range[200]], all of which compute the same result (check for yourself!). However, if I benchmark these commands using RepeatedTiming, I find that 200! runs ~2, ~250, and ~450 times faster than Gamma[201], Product[...], and Fold[...] respectively. This is presumably due to the built-in factorial function knowing some clever tricks for optimizing the multiplication along with not needing to generate and store the list Range[200]. This cleverness of Mathematica can cause some major headaches when trying to benchmark various algorithms. Perhaps algorithm A is in fact better than algorithm B, but my implementation of algorithm B used some built-in function with hidden optimizations, causing me to mistakenly think algorithm B is superior. This can make it quite difficult to separate out the signal (algorithm choice) from the noise (hidden optimizations), necessitating an implementation in a "dumber" language for a fair comparison.

erfink
  • 1,089
  • 6
  • 16
  • 1
    With respect to your notes on SpherePoints[], it might be a good idea to let WRI know that the doc page could be improved. As for the method used, my recollection is that it either uses a point configuration corresponding to a polyhedron (e.g. Platonic) that is already known to be optimal, or uses a phyllotactic approach, similar to what was done here. – J. M.'s missing motivation Dec 10 '21 at 16:11
  • 5
    @J.M. Sure, I could. But what incentive do I have to help a for-profit company improve closed-source code? Especially if I have already reverse engineered a built-in, found it to be suboptimal (cf. http://extremelearning.com.au/how-to-evenly-distribute-points-on-a-sphere-more-effectively-than-the-canonical-fibonacci-lattice/), turned to numerical optimization, and moved on from a toy problem that used it? My institution pays for a license to provide tools, not for the privilege of improving their product. And this built-in is just a single example of the black box problem... – erfink Dec 10 '21 at 23:06
16

I know several people who use Maxima as a widely adopted mathematical computer language on the internet. Some of these people greatly prefer a proprietary language, but they all understand that a proprietary language, which has to be paid for, is no good as a universal language. Because the cost of Mathematica limits the number of people who use it, it will also limit the number of people who will use a software library that uses its language.

To simplify (oversimplify!): if everyone in the world had to pay to speak French, French would stop being a widespread language in double-quick time.

Graham Laight
  • 261
  • 1
  • 4
  • 6
    It is way cheaper to buy a license than to learn a new programming language. It is also way cheaper to use a better tool than a substandard tool. If you can pay a couple of hundred dollars per year and as a result be more productive in something that you do somewhat frequently then it will be worth it. Plus, many workplaces provide licenses for free. My university did for example, rendering the cost argument moot. – C. E. Dec 08 '21 at 11:03
  • 2
    @C.E. learning a new programming language is a one-time investment that may have future returns unrelated to the project that prompted it. — Yes, it is cheaper to use a better tool, but while Mathematica is very good in some ways it is inferior in other aspects. — “A couple of hundred dollars” sounds not much, but this is just where it starts. With a project that is intended to spread to many developers and users over a span of decades, the licensing costs will keep exploding. Mathematica and Matlab like to advertise how easy&inexpensive they are to get started with, but then you're locked in. – leftaroundabout Dec 08 '21 at 11:50
15

I am a professional software developer with a passion for scientific computing, but I’m not about to pay a cent to use scientific computing software when there are comparable alternatives. Hence, I won’t be contributing to the ecosystem in any way, and I’m sure that there are a lot of developers similar to me. In general, in order to promote an inclusive community of users and contributors, software needs to provide some sort of free tier plan. Of course, it seems that WL’s main objective isn’t widespread adoption, but rather commercialization and profit, which, as @Alec Graves touches on in their answer, comes with drawbacks.

Edit:

It has come to my attention that there are some free, though restricted, options for using WL. The first, as mentioned by @flinty is to install it on a Raspberry Pi (wolfram.com/raspberry-pi). The second option, as mentioned by @Jon McLoone, is to use the Wolfram Cloud Basic plan (wolframcloud.com) subject to limitations mentioned here: support.wolfram.com/53990.

I can confirm that both of the above options provide access to WL.

Dean
  • 167
  • 1
  • 1
  • 4
  • Did you read Alec Graves' answer? There is a link there that directly addresses your concern. – polfosol Dec 06 '21 at 18:42
  • @polfosol the article you linked seems to deal with why WM isn't open source. Dean's answer is about Mathematica being a proprietary software with a (quite expensive) license. Alec's answer doesn't seem to discuss either, merely contains the same link. I think Dean has a very valid point here. – Neinstein Dec 06 '21 at 18:56
  • 1
    @polfosol there are some good points in the link, but my answer addresses what I perceive as the main barrier to widespread adoption with WL. My answer should not be interpreted as a renouncement of commercial software. I acknowledge that it has its place. I still can’t see why a more lax monetization strategy can’t be used, similar to Docker, where hobbyists and enthusiasts are welcome to use the language and packages for free, other than the need to maximize profits. – Dean Dec 06 '21 at 19:10
  • 4
    If Wolfram wanted to pursue maximum profit they'd have switched to a 100% SAAS model and made you pay for graphs/statistics/graphics etc as separate modules. Comparable alternatives is debatable. What comparable alternatives?- there are major differences with MATLAB, python, julia. There are significant communities around plugins for paid-for 3D software, paid-for games with modding communities, photoshop, paid-for financial / database / spreadsheet software, and of course Mathematica itself has a community despite being non-free, so it's not generally a requirement to have a free tier plan. – flinty Dec 06 '21 at 23:05
  • @flinty Your first assertion seems baseless. Why would a 100% SaaS model result in maximum profit? Comparable alternatives - well that's just a question of what you're aim is. If you want to run some basic regressions on a slow, Tuesday afternoon, you can probably get away with GNU Octave. If you need a very specific function from a very specific package to avoid a month of grunt work, there is none. The examples you give at the end almost all either have a free-tier option or are services that require some sort of on-going cost for the provider to maintain (e.g. game servers). – Dean Dec 06 '21 at 23:40
  • @Dean SaaS has been a growing trend for ages now and a large number of firms have SaaS apps: you can read this if you want https://financesonline.com/2020-saas-industry-market-report though you don't have to agree it results in maximum profit - clearly many companies are driven to SaaS, so it's hardly baseless. Although perhaps I really meant 'subscription' , as it's becoming increasingly hard to find perpetual licenses for anything - consider VisualStudio which is now subscription - JetBrains, again subscription (but with perpetual fallback), anything Adobe... – flinty Dec 06 '21 at 23:49
  • 7
    @Dean Now I think of it Mathematica already has a free option for hobbyist and enthusiasts by the way: https://www.wolfram.com/raspberry-pi/ , and there's wolfram alpha. Several people on this site have contributed packages for free, e.g IGraphM, or contributed to the WFR, or posted some good answers. If you "won’t be contributing to the ecosystem in any way" then why post? – flinty Dec 07 '21 at 00:09
  • 1
    @flinty I was not aware, I’ll check it out. I just see it risky to learn a language and framework with such a small community. Ironically, I have technically been contributing this whole time. Not to mention, it’s all for the reputation points. – Dean Dec 07 '21 at 00:14
  • 4
    As far as I can see the hobbyist license is not free (https://www.wolfram.com/mathematica/pricing/home-hobby/)? The biggest impediment I see to the general use of systems like Mathematic and Matlab are the runtime isn't free (as far as I understand). I looked into using Matlab at work once but it was prohibitly expensive as a general purpose development system since we had to purchases licenses for each end user along with one per developer. The when you try and integrate with automated build/test systems it gets more expensive again. – David Waterworth Dec 07 '21 at 02:39
  • 2
    @dean Go to https://www.wolframcloud.com/ and click "Sign up for free" this gives you free access to Wolfram Language in the Wolfram Cloud. Limitations of the free account are described here https://support.wolfram.com/53990 – Jon McLoone Dec 10 '21 at 11:00
12

I feel the urge to share my two cents as a software developer and former graduate engineering student:

If you claim that you are able to do everything, most people get the impression that you will fail at most of your jobs, regardless of actually being able to do them.

And that also applies to Mathematica.

I think the big problem is, Mathematica started as a software for symbolic mathematics and then evolved into the Wolfram Language and a tool for everything, I mean literally everything! But it is still under the shadow of its initial reputation. For example, when I was telling my colleagues or my former classmates that I did such and such with Mathematica, and what a cool software it is, most of them were like: couldn't you do it with Matlab's Symbolic Toolbox?

GOD NO.

But the competition is tough. There are several tools for almost every job and convincing someone to try a specific tool for a specific job needs a lot of good reasons including 1) ease of use and 2) not-so-steep learning curvature.

polfosol
  • 952
  • 5
  • 20
  • 1
    i feel like the evaluation mechanism suffers from this. it can be very hard to reason about what a certain piece of code will do. – Michael Dec 07 '21 at 04:06
  • 1
    This aligns with my experience even in physics grad school—the theorists tend to know, but most professors, experimentalists, and undergrads think that "Mathematica" is a thing you use solely to help you do math. They don't really have a sense of WL as a larger, fuller language. – Ben Kalziqi Dec 07 '21 at 17:26
5

So you say the Wolfram Language could be used to program anything. Well, why not prove it. Put up some Youtube videos with pedagogical descriptions on how to do it -- a lot of us learn languages that way. I have lately been learning javascript , react , node-js with express that way (not that i really know it, but starting to get a grip).

Why not describe how to program the client side in a web browser (compared to as example javascript + the react framework). Or make a microserver with REST interface (as in using node-js + express framework) connected to a document database. Or can it actually be done?

These two examples, making the GUI and the microserver, are probably what a lot of programmers are aiming for today. And just perhaps, that is not the main usage for Mathematica, or I might be wrong.

Or if that is not the main usage, put up illustrations where the language really shines. Show us the great things.

Personally, I would be pressed to try it anyway, as I am so used to not paying for my hobby project stuff. Might be different in a large professional organisation (been there before).

ghellquist
  • 151
  • 2
  • 2
    https://www.wolfram.com/broadcast/ – polfosol Dec 08 '21 at 23:00
  • I think open-source community has proofed already that it can be used for anything literally

    regarding the server and client, there is a solution https://github.com/KirillBelovTest/HTTPHandler

    and also, since Wolfram Engine is a freeware tool, so one can add a complete frontend to it https://github.com/JerryI/wolfram-js-frontend

    Then it must be enough to cover all needs of programmers and students, who wants to use Mathematica for free

     – Kirill Vasin Oct 15 '23 at 18:36