Note: the following post may include controversial opinions, so please note that they are only my opinions, and not intended to offend anyone.
I'm being programming in some form or the other since around 1999. I initially used R, and then later, around 2004, mostly switched to Python.
For many scientific applications, for example, simulation, including such things as MCMC, both R and Python are too slow and need to be sped up. The usual way of doing so is by extending with C or C++. For both R and Python, this is what I did, using R's C API with C++, and the Boost Python library with Python.
However, for various reasons, this combination is not the ideal solution. What is important in programming, particularly algorithms? Expressiveness and speed, which are of course related. The more expressive a language, the faster one can write in it.
1) As far as expressiveness goes, neither R nor Python are really ideal for writing scientific algorithms in my opinion. They do not closely map to the underlying algorithm. However, they are both considerably better than C++.
2) I enjoy writing in Python, which is a pleasant language, though as noted above it is not ideal for algorithmic work. However, when one has to work with a Python/C++ combination because of speed issues, this mix becomes considerably less pleasant to work with. What usually happens is that I first write in Python, and once I have something that is working well, often discover that it is too slow (for some subjective value of too slow). I then face the decision of whether to spend some unreasonable amount of time rewriting it in C++, or put up with the slowness. In hindsight I often feel I might have been better off putting up with the slowness, especially as the speedups obtained are unpredictable. Also, the Boost Python interface between the two is a significant maintenance headache, and having code in two very different languages glued together like this is just distracting. No criticism of Boost Python intended, it is as powerful an interface as one could imagine, and pretty much just works most of the time.
Now, in an ideal world, with unlimited time and resources, neither of these problems would be a major deal. However, in scientific projects I have worked on, I've had the following experience.
Whether or not I have collaborators on the project, I always seem to wind up doing the vast majority of the computing. In a total of 5 significant projects, I only had substantial participation from one people on one project. That one person did more than pull his weight; he did as much as me or more. However, in all other cases, including projects with multiple collaborators, I've done (virtually) all the computational work. While I can say that I have not been blessed with the best collaborators (it seems to be a mixture of laziness and incompetence) it is not clear to me whether this state of affairs is likely to change in the future.
Computational scientific work is an enormous amount of effort, and if I can't change how my collaborators behave, I can change the way I work. The most important improvement would be to get things done more quickly. Which brings me to the main consideration here, which is that switching languages to something less orthodox may help. Based on past research, the most likely candidates in order of likelihood are Common Lisp and Ocaml. I've been thinking about this for years, but recently have been thinking about it more seriously.
As far I can tell, few people use either CL or Ocaml for scientific computation. On searching this site, I found two references to CL (one was mine) and one to Ocaml (mine). I've had a couple of encouraging contacts over the years with adventurous people working on the fringe. In 2008 I came across a book review of Peter Seibel's "Practical Common Lisp" (which I own), by Tamas K. Papp. This caught my attention, since it was one of the few mentions of scientific computing for Lisp that I had come across on the net. I wrote to Tamas, who immediately replied helpfully and encouragingly. To quote him
My programming productivity probably increased tenfold with Lisp, but that took about a year to happen and I am still learning (I was doing quite well after 2 months though). So if you are working on something time-critical, then postpone the switch.
You should consider asking folks on c.l.l, I am not the only one who knows about these things, others do scientific computing on Lisp.
He also has a blog and a GitHub page.
Another person I briefly corresponded with (in December 2006) was Ira Kalet, who has used Common Lisp in the context of radiation oncology.
Perhaps there are others who do scientific computing on Lisp, but I don't know of anyone.
The most common problem people cite with CL is the lack of libraries. This is a severe problem in general purpose computing, but may not be so much so in scientific computing, particularly from the ground up implementations of algorithms. Specifically, I can get by most of the time with a basic math library, including probability distribution functions, a multidimensional array library, and a basic set of containers e.g. map, set, list etc. as found in the C++ and Python standard libraries.
I know even less about Ocaml than I do about CL, but threw that in as an alternative. It is supposedly very fast, has one free implementation by French researchers, and seems like the most viable of the ML family of languages for scientific computing.
To conclude, I'm wondering if others have experience with this, and what thoughts they have, if any.
EDIT: I'm mostly interested in first hand experience, in the context of the issues I've discussed above. E.g. if you used to use Python and C++ (or R and C++) and moved to a more obscure language, I'd be most interested in hearing about your experiences.
