13

As a long term user of Mathematica I cannot fail to notice that Wolfram Research seems to have changed their release policy considerably:

https://en.wikipedia.org/wiki/Mathematica#Version_history

For the most part of Mathematica's history you had one release every other year (on average). I liked that because it guaranteed stability, compatibility and well tested functions which worked together well.

Now you have several releases per year, each time with hundreds of new functions (and many bugs as a side effect).

My question
Does anybody know the rationale behind this move? I don't want speculative but qualified answers from people with some background knowledge. Thank you.

Community
  • 1
vonjd
  • 1,595
  • 15
  • 22
  • In theory, shorter release cycles would be a GOOD thing. Shorter release cycles doesn't necessarily mean that you have to accept many new bugs. It does here, in the case of Mathematica, but in general, shorter release cycles does not necessarily mean many new bugs. In theory, quite the contrary: shorter release cycles allow you to be more agile and allow you to push out bug fixes much faster, actually lowering the time of vulnerability. – Andreas Lauschke Jul 22 '15 at 19:43
  • 2
    @AndreasLauschke: maintenance releases, yes ...but not when you add hundreds of new functions each time! – vonjd Jul 22 '15 at 19:45
  • 7
    agreed, but adding bugs with new features/releases is a matter of testing (unit testing and regression testing, and other types), NOT with frequent releases. As I said, frequent release cycles don't necessarily coincide with more and more bugs. Companies like Amazon and ebay are extremely agile and are making hundreds of production releases every day, and the result is getting better and better, not buggier and buggier. Of course, M cannot be re-released that fast, but my point is that increase of bugginess is a matter of testing/QA, not release cycles. – Andreas Lauschke Jul 22 '15 at 19:56
  • 4
    Emerging concept: https://en.wikipedia.org/wiki/Technical_debt. If you think it's bad w/ core functions, wait until higher algorithms like machine learning, data science &c. – alancalvitti Jul 22 '15 at 20:34
  • 1
    @alancalvitti Another concept that has me worried since version 6: feature creep. – Jens Jul 22 '15 at 20:59
  • In the end it still comes down to testing and independent QA. Additing features can be nice, but not untested and "un-QA'ed". Some technical debt is necessary (I think of it as the "low-level plumbing"). Good infra/"plumbing" is a blessing. But it becomes a curse-with-interest if you neglect the core and only focus on new features and then cut corners on testing. – Andreas Lauschke Jul 22 '15 at 21:06
  • @Jens, major transitions going on, eg Cloud, 3D/additive, IoT, sensors, THz, Big Data, etc that affecting all verticals and accelerating in 2015. See eg Emerald ECL1 cloud lab. Think of it as a Cambrian explosion: the world is getting wired as one. Many new features are immediately needed. – alancalvitti Jul 22 '15 at 21:32
  • @AndreasLauschke, WRI is a top-down model but are clearly benefitting from the community: Metcalfe law: crowdsourced identification of bugs and often solutions right here on M.SE. Their tech support can't possibly scale up. – alancalvitti Jul 22 '15 at 21:35
  • @alancalvitti: and your point is ...? I agree with your claim that WRI uses crowdsourcing to find bugs, but I'm allowed to criticize that. You shouldn't be using crowdsourcing to cut corners on testing and QA. And tech support has nothing to do with this, they don't fix bugs. They archive the bug reports and try to answer a few of them. Your last two comments seem rather orthogonal to the original topic, in my opinion. Don't be surprised if I don't respond to your comments anymore in this discussion. – Andreas Lauschke Jul 22 '15 at 21:41
  • 2
    @alancalvitti That's just what I mean: with all those potential playgrounds, trying to do everything, like the kid in the candy store, isn't necessarily a good idea. – Jens Jul 22 '15 at 21:42
  • Well, you COULD do it right. With proper testing and good QA you can manage a lot of moving parts at the same time. You just can't take short-cuts! – Andreas Lauschke Jul 22 '15 at 21:46
  • Vinod Khosla calls it 'technical risk' - like financial debt, technical debt can bankrupt or can be leveraged. Fyi, there are 6 decacorns ($10B valuation) internet firms in US, 2 in Asia, 0 in EU. Unlike the Germans, which only release perfection, Americans and UK take risk and correct later. – alancalvitti Jul 22 '15 at 21:53
  • @AndreasLauschke, it's not orthogonal. What if WRI leveraged the community to do just this? Imagine a globally distributed workforce incentivized to test and QA in exchange for product licence rebate. M.SE is the lever. I will mention this to them at WTC if accepted. – alancalvitti Jul 22 '15 at 21:56
  • @Jens, "trying to do everything" - the common denominator is Analytics language. Provided the underlying algo's are solid, WL represents the efficiency frontier w/ ~5k symbols. – alancalvitti Jul 22 '15 at 22:08
  • @alancalvitti: OK, but in your previous comments you never said that you thought of this as an incentive program. OK, what you mull is perhaps a viable thought, but no more than that: a thought. It will never happen. They know of plenty of very capable people who would like to be alpha or beta testers, and they're turned down. Next, even when you are alpha or beta tester, as I have been several times, they treat you like everyone else: archive the bug reports and perhaps deal with them, or just start reading the next one. External testers don't fix anything -- FIXING is the bottleneck. – Andreas Lauschke Jul 22 '15 at 22:09
  • @AndreasLauschke, let me email you. – alancalvitti Jul 22 '15 at 22:13
  • @alancalvitti: No, I won't continue this discussion. Make your proposal to the conference, as you suggested, good luck. – Andreas Lauschke Jul 22 '15 at 22:13
  • @Jens, since it's topical, re 'microscience', check the range of applications of nanoscribe.de. – alancalvitti Jul 22 '15 at 22:18
  • @alancalvitti There is a long way to go for Mathematica to be able to seriously compete with existing 3D design software. Does it have to compete? I don't think so. – Jens Jul 22 '15 at 22:55
  • 2
    Wolfram did mention this shift in philosophy at WTC 2014. Unfortunately, I don't recall an extensive rationale accompanying the statement, nor did he get many questions about it. I do know users complain when they have to wait a year or so for a bug fix that affects their work. I believe it was implied that WRI wanted to be responsive in a more timely way. Maybe someone else who was there remembers more. – Michael E2 Jul 22 '15 at 23:49
  • a year or so for a bug fix? I've filed bug reports in 2003, that was 5.0 or 5.1, and they were acknowledged as bugs, and they're aren't fixed today. And about ten years ago the director for kernel technology gave a talk on a conf in which he claimed that an analysis of the bug report database showed that about half of the bugs are fixed in 2 weeks (was it ten days?). I don't think these stats still hold ... – Andreas Lauschke Jul 22 '15 at 23:54
  • My sense is that recent, more frequent versions are more buggy but the flip side is access to functionality sooner. Whether you think this is a good or bad idea depends on the extent to which this affects your set-up. My recollection from WTC 2014 was that the rationale was an improved, "streamlined technology stack" allowed this new release schedule although I don't recall anything explicit about weighing a new trade-off between correctness and earlier-access. – Ronald Monson Jul 23 '15 at 00:14
  • 2
    @RonaldMonson: this new "streamlined technology stack" is probably a good thing. If it allows more frequent releases, it's an improvement. But as I said above, if you cut corners on testing and QA, you get ... THIS! The new "streamlined technology stack" should not have anything to do with the thoroughness of testing and QA. If you treat quality as a third-class citizen, you get exactly what we're experiencing with M10 through 10.2. In theory it should be clear that especially with many new features you should intensify the testing and QA efforts! – Andreas Lauschke Jul 23 '15 at 00:23
  • @AndreasLauschke Yes but my point is that with finite resources "cutting corners on testing" might also result in earlier access to new functionality . From my own perspective (and it is my own so I don't want to make too broad a claim about "should") I would still have preferred a 2014 V10 release with all the new Data Science possibilities (functional but yes with some bugs and curious design decisions that have been annoying but not fatal) as opposed to a more watertight late-2015 release. – Ronald Monson Jul 23 '15 at 00:53
  • "Scheisse auf Pizarro" - most comments are past tense like the scribes when Gutenberg came along. – alancalvitti Jul 23 '15 at 04:14
  • @RonaldMonson: the degree to which you do testing and QA is a broad spectrum. If you prefer earlier access and accept more bugs, then that is your preference (not mine), and WRI agrees with you. But then you have what I called "treat quality as a third-class citizen". I don't think that way, but you do and WRI does ... then I think you have less to complain than I have. Lucky you! I agree that early access / lower quality vs. later access / thoroughly tested is a matter of personal preference. I won't disagree with yours. – Andreas Lauschke Jul 23 '15 at 12:59
  • The problem is that we all become beta testers and do WRI's work and I don't like doing other people's work without being paid for it... and thinking about it the situation is even worse because we do WRI's work and still pay them instead of they paying us!!! – vonjd Jul 23 '15 at 14:36
  • 1
    Bingo! You got it! – Andreas Lauschke Jul 23 '15 at 15:46
  • @AndreasLauschke: I saw on your homepage that you are also into quant finance, so Quant.SE might be for you: http://quant.stackexchange.com/ - see you there :-) – vonjd Jul 23 '15 at 17:00
  • @vonjd: part 1: probably not. The broader the topic, the more trolls. I just joined aviation.stackexchange, and I see already how many jackass opinions and "questions" are being "debated" there. Granted, there are also trolls in this group, but the topic is much more specific than "quant finance" or "aviation", so you can expect less trollery in the replies. In some 20 years of quant finance I've encountered way too many clowns to join yet another quant group. – Andreas Lauschke Jul 23 '15 at 18:32
  • @vonjd: part 2: I'm already in about a dozen quant groups on LinkedIn, and even in this professional network you get bombarded with silliness. Way too many people who don't know much and then regurgitate wrong mantras. Not interested in more clown exposure. – Andreas Lauschke Jul 23 '15 at 18:32

0 Answers0