Scoping all symbols

Question

After working Mathematica for a couple weeks to write a larger program, beyond doing integrals, I am starting to gain an appreciation for the Wolfram Language. There are a couple issues that drive me up the wall, and I am looking for solutions.

It comes down to how mathematica scopes, causes lots of bugs. Most programming languages will not let you access a global variable from inside a function, without explicitly asking. This causes two bugs for me:

1)Sometimes I write a block of code, and realize it would be better as a procedure. Often this chunk of code will have global variables in it, because it started out global. When I stick it in a procedure, I need to ensure all the global variables are replaced with the name of an argument to a function. If I miss one, the code usually still runs, but incorrectly, wasting hours of time on long calculations. Most programming langauges would throw an error

2)I define a function, and later add or remove a parameter, the kernel will still remember the old version. If I forget to update every single instance of that function call, it will still work, but on an old version, wreaking havoc on my calcuation.

It there a Mathematica way to solve this problem? Block and Module would not do it, because you have to define all the symbols that are local. Perhaps a block, where you clear all global variables, except functions?

While would be cool is if I could toggle my scoping. For example, at that start of a program I could say EnableDraconianScoping[]. This would raise errors. Once my code it written, I could DisableDraconianScoping[] to reduce overhead that would probably show up doing this for all my functions.

This problem is one of the few reasons I have a hard considering Mathematica for serious projects.

Thanks!

"Most programming languages will not let you access a global variable from inside a function, without explicitly asking." - what? — ciao, Jul 21 '15 at 07:52
You raise a good point. I guess we all develop our own workflows to adapt to this feature. Some combination of ClearAll, contexts, packages, fresh kernels or function definition strategy might prove effective in your development style. — image_doctor, Jul 21 '15 at 08:07
Syntax coloring helps a bit here. You can set globals to have a bright color, and scoped variables a dull color. That way, the globals become prominent within a scoping construct. — J. M.'s missing motivation, Jul 21 '15 at 08:16
Welcome to the world of dynamic interactive languages programming. One replaces the flexibility of coding by more emphasis on the programmer to use their eyes much more to catch errors. In strongly static compiled languages, the compiler can catch many errors or issue warnings, but then one loses the flexibility of quick interactions and the dynamic nature of programming. The problems you describe are not new to Mathematica. Any interactive language will have such problems (Maple, Matlab, Julia, Python, Ruby, Perl, etc...) — Nasser, Jul 21 '15 at 08:23
What should trigger the error you are talking about? I don't know what exactly do you need to catch. p.s. closures are commonly used. Could you prepare a minimal (couple of lines) piece of code you want to modularize in next step and add this feature to? — Kuba, Jul 21 '15 at 08:43
@J. M. There does not seem to be an option for global variables with value assigned in Edit | Preferences | Appearance | Syntax Coloring. — Edmund, Jul 22 '15 at 02:27
@Edmund, you're right. That's where you do a ClearAll["Global`*"] to make them show up (and also has the benefit of removing previous, possibly buggy, definitions). A less drastic approach is Clear[], if you've kept track of the symbols you need to clear. — J. M.'s missing motivation, Jul 22 '15 at 02:39
Now, studying R I see your point, in R the variables are local in the function scope, so you don't have this problem. Unfortunately I don't think this is something solvable in MMA. — Murta, Jul 26 '15 at 03:25

Three Diag · Answer 1 · 2017-08-03T13:29:34.910

Trying to understand how to cope with this exact same problem as almost driven me insane with rage.

How is it possible that the whole of mathematica doesn't have a simple way of scoping routines? How can different packages and routine work if they risk trampling on each other name down to stuff like temp variables name? Ah-ha!

The question contains the answer. The way to scope your functions and prevent either leaks from and to the Global contextm, while requiring only a minimal amount of boilerplate, is to create a package. Here's a simple example.

BeginPackage["MyRoutine`"]

Routine::usage = " Routine[x] Does what you want. In this examples compute a*x^k"
SetupRoutine::usage = "SetupPIA[aa,kk] sets parameter needed to execute routine"

Begin["`Private`"]
SetupRoutine[aa_,kk_]:= (
a = aa;
k = kk;
)
Routine[x_] := a*x^k;

End[]

EndPackage[]

Save the file with the package in your folder under routine.m and load it using << routine.m and call the functions you've defined.

a = 7;
<< routine.m
SetupRoutine[3, 3]
Routine[3]
a

Notice that the parameter a inside your package is prevented from either affecting you current context and viceversa.

In the example above I left the SetupRoutine function exposed, but you could just as easily sweep it inside your routine and pass the parameters to your routine directly.

You can probably read about how this actually work in more detail (for example here or in the documentation, but there's a couple of thing that I understood as I learnt this:

The sequence of BeginPackage[".."] and Begin["Private"] are what does the trick.

I have tried several variations of starting just a context (with Begin[]) and context manipulation but always ended up leaking symbols whenever their name was present in another context.

The func::usage=".." declarations are required.

These declarations specify what functions will be visible outside of the package once you've loaded it. You can go crazy inside the private context with any names, but these ones need not to conflict (possibly with any other package name).

EDIT:

Keep in mind that the above is just a simple example meant to illustrate what is the simplest possible way to scope a function obtaining the feature of the scope of languages like C.

I believe this to be a superior choice whenever one wants to construct a routine which:

Relies on a large list of definitions and parameters which are used across several definitions. This appears to be a common occurrence in scientific computing.
One cannot guarantee that any of the names used inside the routine won't or aren't already used somewhere else or guaranteeing it would require a great deal of unwarranted, tedious and error-prone editing. This is also common occurrence, especially when the code being turned into a routine is in fact an encapsulation of code from a notebook where the main project was being developed.

There are two alternatives to using packages:

Contexts

The issue with contexts is that, without some effort on controlling the context in which symbols are declared for the first time there is always a risk of either leaking variables from Global` to the current context or viceversa.

Consider this:

a  = 3;
Begin["Context`"]
a = 2;
End[]

Due to the way Mathematica handles context parsing I've overwritten the global variable and the scoped variable Context`a is left unspecified. A bit absurd, given that

Begin["Context`"]
a = 2;
End[]
a = 3;

does the opposite. Without actually worrying about either specifying the context to which each variable belong everywhere they appear or meddling with $ContextPath there's no way to guarantee that no matter what there will be no leak of variables definition.

Clearly, in any professional or academic setting I need to be able to rely on my code and make sure that no (possibly hard to debug) error could happen because of shadowing a definition somewhere of say a common variable name such as temp or j happening somewhere in the notebook.

Solutions involving context that guarantee against leaks require a lot of unwarranted typing such as prepending the context name to every single variable which makes for ugly, unreadable code and Mathematica code is not really a beauty to see.

Scoping statements

Block[{x},..] or similar statement can provide local scope to functions but have the obvious drawback of requiring to compile and maintain a list of all symbols that have to be local. This is: tedious, ugly, as it clutters the code of a function with something that doesn't pertain to what the function does, and error prone, since whenever something is changed or added inside the function the list must be maintained.

In general, consider what it takes to answer the question: " how to define a scoping construct which defines the variables locally by default unless otherwise stated " on this very site.

The simplest answer is: packages with their well behaved private scopes!

routine[x_, aa_, kk_] := Module[{a = aa, k = kk}, a x^k] achieves the exact same functionality, with a lot less boilerplate, though. Of course packages are excellent when you have a lot of implementation functions that don't need to be exposed and only one public interface function, but even then pass-by-value, using function arguments, is preferable to modifying a global variable within your package, it seems to me. — MarcoB, Aug 02 '17 at 18:20
Re "How is it possible that the whole of mathematica doesn't have a simple way of scoping routines?" There is Module (as noted by @MarcoB), Block for something closer to dynamic scoping, and With which has a sort of "macro-like" notion. So the question is at best not well motivated, since there is no prior mention of these functions and why they do not qualify. — Daniel Lichtblau, Aug 02 '17 at 18:26
Scoping constructs are ok in this simple example, but end up cluttering the code and require constant maintenance of the list variables. This is a major hindrance whenever there's more than a few definitions to take care of. I will elaborate more from this when I am off my phone. — Three Diag, Aug 02 '17 at 18:42
"Scoping constructs ... end up cluttering the code and require constant maintenance of the list variables"?? That's life in production-level programming, pretty much regardless of language. The Wolfram Language provides some ways to bypass all that, basically by isolating contexts (I think this is done in documentation notebooks so that examples run therein do not affect work elsewhere). But real programming involves real discipline, and those variable lists are a blessing, not a curse, in terms of what benefits they provide for the coder. — Daniel Lichtblau, Aug 02 '17 at 21:38
@DanielLichtblau what motivated this answer on my part is a situation where I have ten definition and a handful of parameters that are common across these definitions. They naturally arose in a context where at some point I needed to encapsulate some of them and be able to turn the parameters into input to a functions.
Defining context itself is not enough, I've tried. But unfortunately if you have a variable in Global, start a Context and create a new variable with the same name, you will shadow the Context one unless you explicitly call it by Contextvariable or swap the $ContextPath. — Three Diag, Aug 03 '17 at 00:18
@MarcoB what do you mean by pass-by-value? I see nothing wrong with specifying global parameters inside the package, all routines have "within" global variables accessibile by all functions within the routine scope, no? — Three Diag, Aug 03 '17 at 01:16
"Pass-by-value" (to reasonable approximation) refers to passing in values of variables to a function in a way that does not retain the names of the values and does not allow those variable values to change outside the function. Also known as "call by value", the most common alternative being "call by reference". — Daniel Lichtblau, Aug 03 '17 at 15:54
@DanielLichtblau can you give a practical example of how you achieve passing by value in the above context? — Three Diag, Aug 04 '17 at 13:16
@DanielLichtblau "That's life in production-level programming, pretty much regardless of language." Sorry to necro, but the only thing I can think of resembling Mathematica's issue in any other language is matching forward declarations in header files with their symbols in compiled libraries, in C and C++. But at least in that case, the linker catches the error before the program starts running, and there's no risk of the wrong symbols being leaked if you mess up — hegel5000, Oct 23 '20 at 16:17
@hegel5000 (1) At this late date, it would be helpful if you describe what you mean by "Mathematica's issue". I was never thrilled with this thread because it struck me as misunderstanding the use-case differences between code modules vs. line-by-line evaluation. (2) If it is too much for comments you can raise the issue as a new question. (3) Is "necromance" really a verb? I guess it should be, given that the noun is "necromancer". I just never thought of that as "one who necromances". — Daniel Lichtblau, Oct 24 '20 at 14:13

Scoping all symbols

1 Answers1

The simplest answer is: packages with their well behaved private scopes!