Context of SQLBinary in Package

Question

I have the following code for a simple key value store for binary data:

BeginPackage["keyValueStore`"];
{open, insert, get, close};
Begin["`Private`"];
Needs["DatabaseLink`"];

open[] := (
   $connection = OpenSQLConnection[JDBC["SQLite(Memory)",""]];
   SQLCreateTable[$connection, SQLTable["data"], {
     SQLColumn["key", "DataTypeName" -> "TEXT"],
     SQLColumn["value", "DataTypeName" -> "BLOB", "Nullable" -> True]
   }];
);

insert[key_, data_] := SQLInsert[$connection, 
  "data", {"key", "value"}, {key, SQLBinary[data]}];

get[key_] := Replace[
  SQLSelect[$connection, "data", "value", SQLColumn["key"] == key],
  {{SQLBinary[data_]}} :> data
];

close[] := CloseSQLConnection[$connection];

End[];
EndPackage[];

which one could use like this:

open[];
insert["one", {1, 2, 3}]
get["one"]
close[];

if you try this, you will find that inserting data will not work but throw a message like:

DatabaseLink`JDBC::error: "Illegal value: DatabaseLink`SQLBinary[{1, 2, 3}]"

now I can repair that by either doing:

BeginPackage["keyValueStore`",{"DatabaseLink`"}]

(which I don't want) or by replacing SQLBinary with Symbol["SQLBinary"] in the code. Both seem to repair a behavior considering the context of SQLBinary that I would consider a bug, but maybe just don't understand correctly what is going on. Would anyone agree that this is not as one would expect and should be repaired? Or can someone enlighten me why this is what I should expect?

But is DatabaseLink`SQLBinary[{1, 2, 3}] illegal or what? What should be the context of SQLBinary? — Kuba, Sep 12 '17 at 09:29
@Szabolcs: that will also not work. It seems to be necessary that either Databaselink` is on $ContextPath or that SQLBinary is created in the current $Context... — Albert Retey, Sep 12 '17 at 09:35
@Kuba: that are exactly the unclear questions. Of course SQLBinary[{1,2,3}] would be correct input given SQLBinary is in the correct contex. When I Needs["DatabaseLink`"] then there is a SQLBinary in the Databaselink` context so that would be the context for SQLBinary that I would expect to work, but obviously in the above situation it doesn't... — Albert Retey, Sep 12 '17 at 09:41
So it looks like DatabaseLink works only when it is in $ContextPath. The reason could be that there's some string-to-code translation somewhere in the package. Maybe through J/Link? A workaround could be to temporarily add "DatabaseLink`" to the context path while your functions is being evaluated (using InheritedBlock). But this still bothers me ... — Szabolcs, Sep 12 '17 at 09:57
If you look in the DatabaseLink package directory, interestingly you'll find a release notes file with a contact email address — Szabolcs, Sep 12 '17 at 09:59
I don't know any Java, so I won't write an answer. But I am pretty sure the reason is this: Look in SQLStatementProcessor.java, lines 23 and 266. What happens is that the Mathematica expression is converted to a string, and this string is checked for correctness by some Java code. If DatabaseLink is in the context path, the string will be "SQLBinary". If it is not, it will be "DatabaseLink`SQLBinary". This doesn't match the pre-defined string SYM_SQLBINARY = "SQLBinary", so the Java code throws an exception. — Szabolcs, Sep 12 '17 at 10:07
@Szabolcs: impressive, what you have detected seems to be the explanation -- and for me looks like a bug, which even could be corrected easily. I think I should file a bug report, would you aggree? — Albert Retey, Sep 12 '17 at 10:10
@AlbertRetey they will probably tell you to always use the second argument of BeginPackage to import dependencies. The problem is that someone who will you your package may now know it. — Kuba, Sep 12 '17 at 10:19
@Kuba and Albert: If they tell you that you must import DatabaseLink in the second argument of BeginPackage, construct an example using this: https://mathematica.stackexchange.com/q/115502/12 In short, if you load your package, which has DatabaseLink in its BeginPackage, then DatabaseLink will be in the ContextPath. If you load a second package, which has your package in its BeginPackage, then DatabaseLink will be loaded, but it will not be kept in the ContextPath. The whole problem is that some DatabaseLink functions work only if at the time when they are called DatabaseLink is in ContextPath — Szabolcs, Sep 12 '17 at 10:28
thanks to both of you, I will file a report. Actually I don't care that much what the answer is as I have workarounds. I just think it makes sense to give feedback, be it considered or not... — Albert Retey, Sep 12 '17 at 19:48

score 5 · Accepted Answer · answered Sep 12 '17 at 10:41

In short, the reason for the behaviour you observe is this:

Some DatabaseLink functions will only work if "DatabaseLink`" is in $ContextPath at the time of their evaluation.

It does not matter how DatabaseLink was loaded. All that matters is whether its context is in $ContextPath at the time when SQLInsert is called.

The detailed reason is that part of DatabaseLink is implemented in Java, and some Mathematica expressions are being sent to Java through MathLink (J/Link). This involves encoding the symbols making up those expressions as strings. Whether they are encoded with or without a context prepended depends on the value of the current $ContextPath and $Context.

If "DatabaseLink`" is in $ContextPath,

Needs["DatabaseLink`"]

ToString[DatabaseLink`SQLBinary]
(* "SQLBinary" *)

If it is not:

Block[{$ContextPath},
 ToString[DatabaseLink`SQLBinary]
 ]
(* "DatabaseLink`SQLBinary" *)

The Java code does some explicit string comparisons of some symbol names with "SQLBinary", and fails if it finds "DatabaseLink`SQLBinary" instead.

The relevant code is lines 23, 110 and 266 of this file:

FileNameJoin[{DirectoryName@FindFile["DatabaseLink`"], "..", "java", 
  "com", "wolfram", "databaselink", "SQLStatementProcessor.java"}]

A possible workaround is to temporarily insert the required context in the context path in every relevant function in your package.

For example,

dbl = Function[expr, 
   Internal`InheritedBlock[{$ContextPath}, 
    PrependTo[$ContextPath, "DatabaseLink`"]; expr], HoldAllComplete];

insert[key_, data_] := dbl@SQLInsert[...]

Pass symbols through MathLink with full context information

thanks for the thorough inspection, I accepted it. It seems now very clear what happens and how we can implement workarounds. But I think you would agree that this has a strong smell of a bug, wouldn't you? — Albert Retey, Sep 12 '17 at 19:45

Context of SQLBinary in Package

1 Answers1

Linked