27

A bit of a loaded question, but in your opinion is Wolfram's team making good progress on improving their software?

I've used Mathematica for almost 10 years now, and honestly I'm personally surprised by how little things have changed since then. I personally see a lot of improvements that can be made to the system and wonder why they aren't done.

What is the community's opinion on the development team's progress as a whole? (Ideally only feedback from people who don't work there, for no conflict of interest).

Maybe there's not enough people using the software or there's not enough demand? I know there can be a lot of incentive problems within tech companies so I was curious what the communities thoughts are on it.

user64494
  • 26,149
  • 4
  • 27
  • 56
Steven Sagona
  • 870
  • 5
  • 15
  • 12
    I feel like with anything complex, the answer is both yes and no. There have been lots of improvements in terms of getting cloud support to the point that people can (almost) host documentation in it. The FE now runs on a 64-bit platform on Mac. There's greater support for stuff like NumericArray and clear interest in improving on that. On the other hand, no matter what your field there are many things where you might want something to have been done, but it hasn't. My big thing is that packaging is still iffy, which dis-incentivizes the community to actually contribute to the system. – b3m2a1 May 12 '20 at 01:36
  • 3
    I'm interested in what you want when you say: "I personally see a lot of improvements that can be made to the system and wonder why they aren't done.". We've all got things we want done, and I'm wondering which angle you're coming at this from. – b3m2a1 May 12 '20 at 03:44
  • 1
    @b3m2a1, I'm in physics and while I use and love mathematica, it's a sort of "fringe" thing to use. If you need something done by a computer you use Matlab or Python if you want to sound cool. Everyone works out math by hand. Personally, I think that Mathematica's ability to do symbolic calculations is completely under-utilized, and it's in-part because no development has been made to improve it in 10 years. Wolfram Alpha is sort of unbelievable in college when you want to do differential equations, but I've never even /heard/ of anyone using Mathematica for differential equations in my field. – Steven Sagona May 12 '20 at 20:51
  • 3
    I think this question belongs on the main site. Meta is for questions about Mathematica Stack Exchange. – Simon Woods May 14 '20 at 20:09
  • @StevenSagona I agree completely with you here. Even as a post graduate student I have seen people use Mathematica very little. Although whenever I visit this SE I am amazed by the potential of this software. Even I have got analytical solutions, I would have never gotten by hand calculations. – Avrana May 16 '20 at 08:03
  • 3
    @StevenSagona At my institute (physics) people use it a fair bit for symbolic stuff. Actually, I would have said that the issue is that people tend to use it as a fancy symbolic calculator or plotter, and ignore the other abilities. It has always been my impression that physics has been a traditional stronghold of Mathematica. In other areas of physics (that they don't do at our institute) it is even more popular, see hepforge – Szabolcs May 16 '20 at 08:11
  • 4
    @b3m2a1 "My big thing is that packaging is still iffy, which dis-incentivizes the community to actually contribute to the system." <-- This, 10 times. There are multiple projects that started out on Mathematica and moved fully to Python, e.g. xcellerator. I think this is the main reason. – Szabolcs May 16 '20 at 08:16
  • 1
    I can only speak for myself, but until version 5, there were no sparse arrays, and until version 10, there were no associations. Those two data structures readily come to mind as developments that certainly made it easier for me to implement algorithms I'm interested in. – J. M.'s missing motivation May 21 '20 at 11:18
  • 1
    In many ways WL is accelerating as shown by the ability of WRI - a private firm of ~800 employees - to add and combine features from so many domains (eg anatomyData, 3D modeling, geolocation, data structures for data science more advanced than python/pandas, Emerald's SLL...). But because it's a private language it only improves along directions dictated by owners w/ limited resources, rather than organically from network effects as in open source systems, so you get internal poaching of devs and initiatives stall or crash and burn. WRI needs to improve their biz model first & get the revenue. – alancalvitti May 21 '20 at 14:20
  • 1
    Mathematica development feels more nimble than its peers -- Matlab and Maple. Cloud, GPUs for computation, natural language interfaces are just a few examples – Yaroslav Bulatov Sep 08 '20 at 21:23

6 Answers6

29

Update January 19, 2024. added V 14 integration result (differential equations and PDE's update for V 14 will take more time to complete)

Update July 12, 2023. added 13.3 integration result (differential equations and PDE's update for V 13.3 will take more time)

Mathematica is very large and complex software which covers many many different areas.

Some of the things that WRI needs to improve on is overall quality and performance of the FE and or the kernel, to make things more robust and less prone to hangs, crashes and sudden freezes, specially when using Dynamics.

I can only comment myself on few specific areas. In the area of integration, this bar chart shows the progress made. This is for integration test using 14,944 integrals of many types. These integrals are a subset of Rubi's full integration test suite.

Integration

The percentage solved remained the same for V 14 as V 13.3.1, at 98.68% but the grading and leaf size improved. It is probably hard to improve much when % solved is as high as it is now.

Note that the percentage solved have been corrected for all systems to count integrals that were not solved but are known to be not integrable as passed instead of failed as before. The Rubi test suite has number of integrals that are not solvable.

This now match the way the Full CAS integration test count things.

enter image description here

The quality of the anti-derivatives has improved in V 14. This chart shows the percentage of A grade of anti-derivatives for same test over the years. A grade means optimal anti-derivative. (Higher is better)

enter image description here

There are 4 grades for each integral. A,B,C and F. Where A is best, and F for not able to integrate it.

enter image description here

The leaf size also improved.

enter image description here

So overall, I would say that Integrate has been improving over the years.

DSolve for PDE

For DSolve in the area of PDE's, using test suite of about 2,000 PDE's, here is the result

Mathematica graphics

This shows steady and good improvement in this area.

DSolve for ODE

In the area of DSolve and ODE's, using the famous Kamke's set of ODE's (1,940 differential equations from Kamke book), this is the result

Mathematica graphics

This also shows steady and good improvement in this area.

Some of the things that WRI needs to improve on is overall quality and performance of the FE and or the kernel, to make things more robust and less prone to hangs, crashes and sudden freezes, specially when using Dynamics.

Oh, and I forgot one thing we all need and been waiting for for many many years: an easy to use debugger !!

Edit

Additional related charts on this subject are at

Differential equations, version 12.3 and improving the level of the questions

lericr
  • 27,668
  • 1
  • 18
  • 64
Nasser
  • 143,286
  • 11
  • 154
  • 359
  • Great response. Thanks – bbgodfrey May 15 '20 at 01:53
  • 1
    I think this is fair.

    One comment here is that I'm honestly not sure how many of these solutions are actually useful. While technically Mathematica can do a lot, we're at a point were even simple analytical problems can end up being obfuscated. You might get a solution to your differential equation but if its a nasty long expression that can't be written on the page, then it defeats the point.

     – Steven Sagona May 12 '20 at 21:03
  • You often solve DEs with Maple that cannot be solved with Mathematica. In your judgment, which works better for symbolic solution of DEs? – bbgodfrey May 15 '20 at 01:38
  • 4
    @bbgodfrey for analytical solutions of DE's, Maple still has small edge. For Murphy's collections of ODE (2,313 ODE's), Maple 2016 scored 95% and Mathematica 11 scored 85% (this needs to be updated to current version). On Kamke's collection, Maple 2020 scored 92% and Mathematica 12.1 scored 84%. But DSolve has been improving by few percentages over each release which is good. Also Maple's ODE solver still has more options and Misc. functions than DSolve has. – Nasser May 15 '20 at 01:48
  • 3
    +1. Though if you used the same range [0, 100%] over the y axis of your histogram (https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics), someone could argue that indeed little progress has been made e.g. for integration :-) – chris May 16 '20 at 07:21
  • I'm a bit surprised that Maple is ahead of Mathematica. If that really is the case, and Maple is open source, then it really suggests to me that these improvements are not that very impressive. Good that the software is improving of course, but not that great. – Steven Sagona May 19 '20 at 07:39
  • 1
    @StevenSagona Maybe you're thinking of SageMath? As far as I'm aware, Maple is proprietary software, not open source. – MassDefect Jun 09 '20 at 23:09
  • 1
    I have two comments, based on both the original note and the recent update. (1) Yes, Integrate has improved of late, both indefinite and definite. Of those, I'd say indefinite is the less important, and neither is really all that important in the grand scheme. I write this from the perspective of a developer who has been heavily involved in these improvements... – Daniel Lichtblau Jul 12 '23 at 16:50
  • 1
    ...(2) You stated "Some of the things that WRI needs to improve on is overall quality and performance of the FE and or the kernel, to make things more robust and less prone to hangs, crashes and sudden freezes, specially when using Dynamics." You could have begun the note with this point. This set of issues is extremely important, and extremely difficult. It's a thicket where I have on occasion even sleep. Again, this is from the perspective of one with heavy involvement in the development. – Daniel Lichtblau Jul 12 '23 at 16:54
7

To be quite honest, I'm increasingly disappointed by Mathematica's progress. Yes, some of the core functionality has improved in efficiency, but there are still gaping holes in what you'd think would be important core areas, things that are important to many different fields.

For a key example, consider FullSimplify itself. FullSimplify has had a bug where just renaming variables can change its output in fundamental ways (sometimes up to completely breaking its functionality) for at least six years; according to its own documentation it hasn't been updated since 2014, a solid two and a half versions ago. It behaves weirdly and inconsistently with assumptions, and can't seem to handle custom ForAll rules as generally as the documentation claims. It's very hard to argue that this is a niche interest, too, since it's one of the core functions of any CAS.

Graphs were introduced in 2010, but up until very very recently have had almost no actual support; I remember as late as two years ago finding functions that straight-up didn't do what the documentation said they did and behaving oddly any time you weren't dealing with a simple graph (i.e. anything with multi-edges). Symbolic vector and tensor manipulation is also oddly primitive for a CAS; Mathematica doesn't even understand the distribution law for dot products, and thanks to the poor performance of FullSimplify it's difficult to implement such laws yourself.

Meanwhile, there's been entire new packages added for things like neural networks, which Mathematica is fundamentally poorly built for not least because of the above limits on simplification and symbolic tensors. As a big fan of functional programming I'm somewhat in support of their new functional programming packages -- but it, and the neural net thing, and the recent cloud support, really makes me feel like Wolfram has just been chasing the latest fad for years instead of actually focusing on making their core product any better.

linkhyrule5
  • 599
  • 3
  • 10
  • 1
    The behavior noted for FullSimplify is in the nature of the beast. It is not a bug. As for ForAll, possibly the better functions to use in further manipulations would be Resolve or Reduce. Regarding neural networks, I am not sure how improvements in symbolic simplification and/or working with symbolic tensors might be of benefit. – Daniel Lichtblau Sep 27 '21 at 14:52
  • It's a bug. It doesn't do the predictable thing, and it doesn't do the thing it's intended to do, namely reliably simplify expressions. Even if it was some fundamental difficulty of the problem, it'd still be a bug. But also, it's really not. There's a five second patch job -- just canonicalize your variable names first, then at least you have deterministic output that doesn't depend on semantically irrelevant parameters. – linkhyrule5 Sep 28 '21 at 07:55
  • Reduce and Resolve operate on equations and relations, and as such are simply not the right tool for the job. I want to simplify expressions, not reduce propositions. – linkhyrule5 Sep 28 '21 at 08:01
  • 1
    (1) I was providing the information so you could decide whether and how to work with the software. If you choose to fight with it, that's fine too. Regarding Resolve and the like, I mentioned those because they are used in handling quantifiers such as ForAll. It might happen that members of the Simplify family also handle it at times, but that would be incidental rather than fundamental to what they are designed to do. – Daniel Lichtblau Sep 28 '21 at 13:41
  • 1
    (2) For the result of Simplify and family to be both no larger than the input, and also a bug, it would need to be, at a minimum, mathematically different from the input. Even then some exceptions are allowable in the framework of what they do. A fundamental tenet of this functionality is "You get what you get". Canonicalization, whatever that might mean, is not part of the bargain. – Daniel Lichtblau Sep 28 '21 at 13:44
  • 2
    (3) "Five second patch job"? I have to say that one brought a smile this morning. (Might have been a smirk, I don't have a mirror handy.) Point being, I've done this stuff for a while, and I actually get paid to figure out what are the tar pits to avoid. In a nutshell, this approach will fall short because it is the variable ordering rather than naming that matters. – Daniel Lichtblau Sep 28 '21 at 13:49
  • Uh, no. You use Simplify to simplify expressions. You can play word games all you like, but if it doesn't do the thing you actually/practically want, it's a bug. I don't care if it's not mathematically distinct; I'm not some logically omniscient hypothetical oracle that doesn't care about the form of its inputs, I want a comprehensible, simplified output. – linkhyrule5 Sep 28 '21 at 18:02
  • Also, you're just plain wrong. You can fix the problem by changing the names, therefore you can fix the problem with surprising/inconsistent behavior by standardizing the names. The underlying bug may or may not be due to variable ordering, but that's not the point here; the point is that there is a nontrivial practical benefit to the end user gained by using standard variable names internally. – linkhyrule5 Sep 28 '21 at 18:04
  • 1
    (1) "Also, you're just plain wrong." Of course. What would I know about it? I mean, other than that variable substitution is tricky (due to possibility of nesting, as well as the "which ones are variables?" question being nontrivial), and that the outcome is dubious. I write from the perspective of someone who actually has implemented variable substitution. In multiple places, and over a period spanning three decades. – Daniel Lichtblau Sep 28 '21 at 21:14
  • 2
    (2) The Simplify family is not a set of mind-reading functions. If it does not do what you want, the best practical advice is to use different functions. Insisting it is a "bug" is not going to help get your work done. – Daniel Lichtblau Sep 28 '21 at 21:16
  • 2
    (3) Playing word games? No. I am simply explaining that a certain behavior is not a bug. And, apparently, wasting my time in so doing. Over and out. – Daniel Lichtblau Sep 28 '21 at 21:18
  • 3
    (Against my better judgement...) Some background and a variable ordering workaround – Daniel Lichtblau Sep 28 '21 at 21:42
  • Very, very belated, but... that answer is nonsense. All you have to do to standardize the behavior of FullSimplify is to sort the variables into canonical order first, which is a well-known approach in standard use in functional language compilers. I am well aware that the result may or may not, in fact, work, but it does at least reach the much lower bar of being predictable and comprehensible; the result is to change obscure Heisenbugs into just plain deterministic bugs that can be consistently expected and assessed. And that is already a huge improvement. – linkhyrule5 Aug 25 '23 at 09:39
  • Apply your reasoning to this simple example. Simplify[x^2 + 3*a*b + y^2, Assumptions -> 3*a*b - 2*x*y == 0]. Then indicate how it might give a similar result when the variables change using the replacement {a -> x, x -> a, b -> y, y -> b}. To make that explicit, I mean rep = {a -> x, x -> a, b -> y, y -> b}; Simplify[x^2 + 3*a*b + y^2 /. rep, Assumptions -> (3*a*b - 2*x*y == 0 /. rep)]. – Daniel Lichtblau Sep 10 '23 at 17:23
  • Sure. In the first case, the first thing it does is reorder the variable names in order of appearance, so the first becomes Simplify[a^2 + 3 b c + d^2, Assumptions -> 3 b c - 2 a d == 0]. Then it does whatever it does, which I do not specify in my suggestion. – linkhyrule5 Sep 11 '23 at 04:46
  • In the second case, since Simplify is not HoldFoo, it first applies the replacement, giving Simplify[a^2 + 3 x y + b^2, Assumptions -> 3 x y - 2 a b == 0]. Then, it reorders the variable names in order of appearance, so it becomes Simplify[a^2 + 3 b c + d^2, Assumptions -> 3 b c - 2 a d == 0]. It then does whatever it does, which gives the same result as in the first case. – linkhyrule5 Sep 11 '23 at 04:48
  • The result may or may not be useful in this form, but at least it would be predictable, and not depend on semantically irrelevant factors. Ideally, we would then proceed to provide more options and functionality so as to permit actually useful results, but at least we wouldn't get surprised by obnoxious unintuitive bugs. – linkhyrule5 Sep 11 '23 at 04:51
  • The point of my simple example, which you should seriously try rather than guess, is not that a result comes out that's more complicated than desired. It is rather that one gets two structurally different results based entirely on variable names in the input. Ergo, not predictable. This is due in part to canonicalizing of expressions, wherein x^2 + 3*a*b + y^2 evaluates to 3*a*b + x^2 + y^2. so one will not get the claimed a^2 + 3 b c + d^2 on reordering (using any sensible reordering, at least). And x^2 + 3*a*b + y^2 /. rep evaluates to a^2 + b^2 + 3*x*y, again counter to the claim, – Daniel Lichtblau Sep 11 '23 at 15:40
  • If you want to be taken seriously, take the example seriously. Trace through the basic inputs, see what Simplify is actually getting as input. Then try your strategy. Also tamp down on the rude tone. It's simply not appropriate for a serious discussion. (If you only want to play the troll, that's a different matter). – Daniel Lichtblau Sep 11 '23 at 15:44
  • I am being emphatic because it seems we have fundamentally different expectations for what sorts of things people should do to fix their product. My argument from the beginning has been that Simplify, and FullSimplify, are sufficiently fundamental and important functions that much, much more effort should be done to make them work. – linkhyrule5 Sep 13 '23 at 13:32
  • To that regard, it really doesn't matter how Mathematica currently does things. I told you how, under my implementation and in accordance to the semantic rules Mathematica claims to operate under, the expressions should evaluate. If they do not in fact evaluate in that way, that just means more things need to be changed. (In this case, I don't know why, if you were recoding Simplify from the ground up, you would apply a canonicalization function on the expression when the proposed variable renaming system is also a canonicalization function. As I expressed in the original.) – linkhyrule5 Sep 13 '23 at 13:33
  • You seem to have missed the implied Yiddish. You have not in any way shown how to get syntactically equivalent results for the two variants in my example. There is of course a reason you cannot show it. But apparently you haven't even tried. What you have made clear is that this you are proposing something that you simply fail to understand. Find a new topic. – Daniel Lichtblau Sep 13 '23 at 15:18
7

My answer would be (Feynman's famous question): "Compared to what?"

I'm fully immersed in the symbol-manipulation power of Mathematica and while I haven't studied other systems in any depth, I'm immensely impressed by the powerful functions and kinds of problems that are now amenable to symbolic computation.

The curated databases continue to grow.

My only real complaint is that it is still awkward and difficult to search for and through curated databases. Mathematica's natural language interface is simply not up to the job.

Example: I wanted to determine whether there was a database of reflectance spectra of common surfaces (a leaf, paper, apple, etc.). I consider myself a superb Mathematica programmer with decades of experience, nevertheless I took quite a long time searching and never found what I was looking for. Frankly, I still don't know if I simply missed it, or whether it doesn't exist. How does one know??

Nevertheless, I'm a complete devotee to the language and wouldn't switch for the world.

David G. Stork
  • 41,180
  • 3
  • 34
  • 96
  • I mean, that's not really a useful answer IMO. Mathematica is a commercial product which has had years to improve and many major versions; the question isn't "is it better than its competition", but rather "is it as good as it could be, given the amount of resources we pay for it." Not least because the former has horrible incentives towards just crushing your competition instead of improving your own product... – linkhyrule5 Sep 28 '21 at 08:00
  • @linkhyrule5: Well, to be more specific, then, I'll say Mathematica is IMPROVING faster than the competition, and has consistently led the competition in new functionality. It was (and is) FAR ahead of its competition in symbol manipulation, it pioneered "computable knowledge" (as distinct from search). Tell me: what software system do YOU think "is as good as it could be given the amount of resources we pay for it"? My own experience concerning COST (which looms large in your mind), is that even a SINGLE function that saves me one day of processing more than pays for the cost. – David G. Stork Sep 28 '21 at 17:35
  • shrug Honestly, I'm just not in the habit of making excuses for large corporations with multimillion-dollar budgets. I'm fundamentally not happy with "good enough", or "better than nothing", because I (and others) have to keep paying Wolfram for new versions. – linkhyrule5 Sep 29 '21 at 12:13
  • 1
    Nobody is asking for you to "make excuses." I asked if you think anybody is doing better than WRI... and apparently you can't think of anybody. And do you have any idea how difficult it is to develop a system like Mathematica? (or Matlab? or ....) – David G. Stork Sep 29 '21 at 16:12
  • I don't think anyone is doing better, no. I also don't think that question is particularly relevant. I don't think "see if anyone else is doing better" is a good standard, because among other reasons it's very easy for a powerful corporation to make sure of that by predatory business practices. It's precisely because I don't know how difficult it is to develop a system like Mathematica that I don't really trust the market to tell me. – – linkhyrule5 Sep 30 '21 at 01:23
  • 1
    @linkhyrule5: Wolfram Research is very much NOT a powerful corporation involved in predatory business practices. So many other languages are more popular than Mathematica. I suspect Matlab (to take just one example) outsells Mathematica 10:1. – David G. Stork Sep 30 '21 at 02:28
4

I use WL almost daily in a variety of roles, mainly focused on prototyping data processing or signal processing applications. I go back a fairly long way with Mathematica to V4.1 so can say it has improved steadily across many, many areas. To contrast, I also use Matlab, R and Python depending on the end use and expected production environment. WL and Mathematica are my first go-to for problem solving but I generally end up with other end environments for production use. The reasons vary, but many times the deployment cost and integration with other computing environments is generally more favorable with non-WL environments.

A couple points that have irritated me are the continuing addition of functions which roll-up or specialize existing functions. The functional churn on small-scale changes is problematic for continuity of understanding and applying WL. The addition of arcane functions is also a continuing irritant when the main functions need optimization for speed and deployment - one V13.3 example would be RegionHausdorfDistance - how many WL users really needed this function when those Wolfram resources could have been spent elsewhere? I am certain a couple users were happy to see that function, but at what cost in overall software bloat and complexity? The breadth of WL is its great strength and weakness at the same time.

The help system also could use some restructuring in the absence of any real documentation. I find myself running through loops of links which are tangential to getting direct help for a syntax issue. The increasing appearance of 'hidden' syntax connections documented in forums like this is also occasionally vexing.

Would I quit WL and Mathematica? - no, but I am exhausted by the functional churn. Sometimes I get lucky and there is a new function or feature which actually helps, but most of the function churn at each new version will never be used by me.

Lexington1776
  • 619
  • 1
  • 6
3

I use Mathematica daily. I couldn't live without it. It's one of those things until you have it you don't know what you are missing and when you do you cant do calculus by hand anymore. I just wrote code in Mathematica that outputs Indian classical music pieces on its own. If that does not impress you, nothing will.

Quasar Supernova
  • 1,678
  • 8
  • 13
  • 5
    It would be really cool if you needed to self-answer a question you asked and had to share that code :O – CA Trevillian May 16 '20 at 15:49
  • 3
    This doesn't address my question at all. I understand you use Mathematica daily, but the question is if you think Mathematica has adequately improved over the years. For example, are you using features that we added in the last 10 years? Have there been any new versions that you've been happy about? Have the new mathematica versions actually changed your mathematica experience at all? – Steven Sagona May 19 '20 at 07:30
  • yes to all the questions. The latest version is the best. – Quasar Supernova May 22 '20 at 03:35
1

I'm a former derivatives trader, and an amateur user of WL. I've been around since v11, after Association came about, which seems to be the most substantial development in quite a while. While I have no doubt many parts of the language are becoming more optimized, I haven't witnessed any breakthrough functionality. I enjoyed WebExecute (12.0) and the new bracket/brace/etc. formatting from AutoOperaterRenderings (12.3), but both of those are quite trivial.

For the gurus here, I know there are endless problems to be fixed that you have complained about for years. But if overnight those problems were solved, would anything really be different? Sure the bugs would be fixed, but I don't think that means the language has evolved. The rest of the world does not care about WL. And if a true breakthrough really did come about, would anybody besides the people here care? What good is improvement in the language without growth in adoption?

Nevertheless, I love the language, and I have no desire to ever use anything else again.

kits
  • 270
  • 1
  • 7
  • I mean, the people here are using Mathematica for things like engineering, biochemistry, fundamental physics. So yes, yes people would care, at least if they were aware of the full scope of the consequences. – linkhyrule5 Sep 28 '21 at 08:04