35

What's the difference between these two ways of specifying dependent packages?

Method 1:

BeginPackage["foo`bar`", {"xxx`", "yyy`"}]

Method 2:

BeginPackage["foo`bar`"]
Needs["xxx`"]
Needs["yyy`"]

Do they always have the same effect? Note that these packages may be used in a big collection of packages which have complex interdependency.

rm -rf
  • 88,781
  • 21
  • 293
  • 472
user13253
  • 8,666
  • 2
  • 42
  • 65

2 Answers2

29

While @rcollyer has answered the "what", I feel that this question still deserves an additional answer because no less important is IMO the why.

Public and private package import

The first form (contexts in BeginPackage) is called the public import. What this means is that, in addition to making the public symbols of those contexts available for the implementation of your package, you also make them available for the code that loads your package, be it some top-level code or another package. The two typical use cases when you may want to do this are:

  • You want at least some of the functionality of the packages you use in your package to be also available to the end user
  • Your package is actually an extension of some other package, much like a subclass extends its parent class in OOP.

The difference between these two scenarios is sometimes blurred.

The second form is called private import, and is recommended for most cases when you import some functions from other packages into yours. Most of the time, you want it this way, since you only use those functions in implementation of your package's functions, and otherwise the end user could not care less what they are.

How it works

Technically, the encapsulation is realized by the way how BeginPackage and EndPackage functions manipulate the $ContextPath. What happens is that BeginPackage calls Needs on all contexts indicated in its second argument (which is why the ordering is different, as @rcollyer indicated), so that the corresponding packages are loaded (if necessary) and the contexts added to the $ContextPath in such a way that they will not be removed by EndPackage. OTOH, all the modifications of $ContextPath in the body of the package between BeginPackage and EndPackage are undone by EndPackage, which makes the packages imported by calls to Needs in the package body to be privately imported.

An additional subtlety

There is one additional subtlety related to the use of BeginPackage, which is not widely known and can be puzzling at first. Consider some package A` , which imports packages B` and C` publicly (in other words, it starts with BeginPackage["A`",{"B`","C`"}]). Imagine that A` has been loaded, but is not on the $ContextPath at the moment (for example, it was privately imported by another package). Then, if you call Needs["A`"], not only will A` be added to the $ContextPath, but also B` and C` .

This is a rather natural behavior when we think about it, but back in the day it took me a while to figure out why calling Needs on a single context brings many contexts to the $ContextPath. What this means is that the public dependencies declared through BeginPackage are cached. This does not happen for private package imports.

Further information

A much better and more complete description of these mechanisms can be found in the book of Roman Maeder "Programming in Mathematica", which IMO remains to this day the best reference for package mechanics in Mathematica.

Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
  • 1
    Likely the mechanism is implemented via Begin[{$ContextPath}, ...] (or similar) and Needs is either called before it (public import), or within the body (private import). So, that when $ContextPath reverts to its original value, any package loaded privately is removed from the $ContextPath, also. – rcollyer Nov 01 '12 at 05:15
  • 2
    @rcollyer Yes, could be. One can surely imitate the action of BeginPackage and EndPackage by the top-level code, that's what I think is really important - we are given a great deal of control over encapsulation, which is what I think is a right design. – Leonid Shifrin Nov 01 '12 at 05:26
  • +1 For the important usage guideline of generally using the second form unless necessary. I like the name "private import" too. – Andrew Moylan Nov 01 '12 at 05:48
  • @AndrewMoylan Thanks, Andrew. The name is not mine though, it was used by Roman in his book. – Leonid Shifrin Nov 01 '12 at 06:01
  • 1
    I agree with you @LeonidShifrin that Maeder's books are still the best in many respects. I only wish they would be updated/expanded to cover the current MMA version. They were written in the 90's and cover versions 2.x and maybe 3.0. Yet they are still very relevant due to the smooth evolution of MMA in the following 20 years. – magma Nov 01 '12 at 10:23
  • Although, encapsulation is leaky in packages. All symbols are ultimately accessible. – rcollyer Nov 01 '12 at 12:52
  • @rcollyer I think this is a good thing (leaky encapsulation), on a large scale at least. The reason I think so is that the other side of it is the level of control over the encapsulation given to the end user. For good programmers, more control is always good, and bad programmers will write bad code in any language. This is IMO different from the leaks of lexical scoping (small scale encapsulation), which are more worrisome. – Leonid Shifrin Nov 01 '12 at 17:23
  • @magma Yes I agree. I had a chance to chat with Roman during this conference, and mentioned this. My impression was that he wouldn't mind doing that in principle, but is busy like anyone else is these days. – Leonid Shifrin Nov 01 '12 at 17:26
  • @LeonidShifrin I always wondered if he was still actively involved with WRI. Do you know if he still is? I do not remember having seen his name mentioned recently. – magma Nov 01 '12 at 22:04
  • @magma He has been (and continues to be) involved in the parallel programming functionality of Mathematica. – Leonid Shifrin Nov 01 '12 at 22:05
  • @LeonidShifrin since you mentioned the words PrivateImport and PublicImport, I thought you might help me with a question regarding a few properties in the newer Paclet System's PacletObject[...]. There is a property called "HiddenImport" and "PrimaryContext". Can you explain what they are and how they work? – user13892 Feb 07 '24 at 15:31
  • @user13892 That merits a full-fledged separate question to be asked on the site. I can't promise to answer that right away, but can try to do that soon enough, and if I don't, it would allow others who can, to answer that too. – Leonid Shifrin Feb 14 '24 at 12:33
26

A little experimentation reveals the differences. Including the packages within BeginPackage:

In[2]:= $ContextPath
    BeginPackage["Temp1`", {"Quantum`RealHarmonics`"}];
    $ContextPath
a = 5;
EndPackage[]
$ContextPath
(*
Out[2]= {"Cell$$10824`", "System`"}

Out[4]= {"Temp1`", "Quantum`RealHarmonics`", "System`"}

Out[7]= {"Temp1`", "Quantum`RealHarmonics`", "Cell$$10824`", "System`"}
*)

Using Needs after BeginPackage:

In[2]:= $ContextPath
    BeginPackage["Temp1`"];
    Needs["Quantum`RealHarmonics`"]
    $ContextPath
a = 5;
EndPackage[]
$ContextPath
(*
Out[2]= {"Cell$$10826`", "System`"}

Out[5]= {"Quantum`RealHarmonics`", "Temp1`", "System`"}

Out[8]= {"Temp1`", "Cell$$10826`", "System`"}
*)

So, the ordering of the contexts is effected which effects how different symbols are resolved. Additionally, Quantum`RealHarmonics` does not make it into the global $ContextPath once EndPackage is run. The second form will then not pollute the global context, unlike the first.

Note, that while they do not "pollute" the global context, i.e. you cannot refer to them by their unqualified name, they are still loaded and can be referenced by their fully qualified name. For example, Quantum`RealHarmonics`RealHarmonicsMatrix` would work in the second case, but RealHarmonicsMatrix would not after EndPackage[] has executed.

rcollyer
  • 33,976
  • 7
  • 92
  • 191
  • For the curious: yes. Quantum`RealHarmonics` is a real package in my collection. I picked it at random which, incidentally, loads a package Utilities`Matrix` via the second method (but in the private section) to avoid polluting the global context. – rcollyer Nov 01 '12 at 03:55