Held keys in associations

Question

Bug introduced in 10.0 and persisting through 11.0.1 or later
Bug introduced in 10.0 and fixed in 10.4

Association is new in 10.0.

Bug still present: AssociationMap[Hold, p]
Fixed bug: Hold /@ p

---

Jacob Akkerboom commented:

Note that PositionIndex does work correctly with held expressions, whereas this is a bit painful to implement using GatherBy. a = 1; PositionIndex[Unevaluated[{a, b, c, d}]]

I finally got around to exploring this behavior and I am a bit confused and troubled by what I see.

First Jacob's example with a more explicit definition of a:

a = "Fail!";
p = PositionIndex[Unevaluated[{a, b, c, d}]]

<|a -> {1}, b -> {2}, c -> {3}, d -> {4}|>

We see from this that Association is capable of containing unevaluated key names. We can also extract the value corresponding to the key, or use the Association in a replacement:

p[Unevaluated @ a]

Hold[a, b, c] /. p

{1}
Hold[{1}, {2}, {3}]

Other operations prove problematic however. In the first and third cases a evaluates undesirably:

Hold /@ p
KeyMap[Hold, p]
AssociationMap[Hold, p]

<|"Fail!" -> Hold[{1}], b -> Hold[{2}], c -> Hold[{3}], d -> Hold[{4}]|>
<|Hold[a] -> {1}, Hold[b] -> {2}, Hold[c] -> {3}, Hold[d] -> {4}|>
Association[{Hold["Fail!" -> {1}], Hold[b -> {2}], Hold[c -> {3}], Hold[d -> {4}]}]

Since KeyMap does work I thought we could use HoldPattern for the keys, but that too has problems in that the original lookups and replacements no longer work:

p2 = KeyMap[HoldPattern, p]

<|HoldPattern[a] -> {1}, HoldPattern[b] -> {2}, HoldPattern[c] -> {3}, HoldPattern[d] -> {4}|>

p2[Unevaluated @ a]
Lookup[p2, Unevaluated[a]]
Hold[a, b, c] /. p2

Missing["KeyAbsent", "Fail!"]
Missing["KeyAbsent", Unevaluated[a]]
Hold[a, b, c]

The lookup does work if we use an explicit HoldPattern but that seems somewhat undesirable:

p2[HoldPattern @ a]

{1}

To make the replacement work we can convert the Association to a Dispatch expression:

Hold[a, b, c] /. Dispatch[p2]

Hold[{1}, {2}, {3}]

My questions:

1. Is there a better way to work with unevaluated keys in an Association?

2. Is the illustrated behavior intended and desirable?

3. Are unevaluated keys an expected application of Association? The behavior of PositionIndex indicates it may be but the conflicting results of the mapping operators seem unplanned.

Answer 1

1. Wrap the entire key in Hold, if you really want to keep things that might evaluate in the keys of an association (HoldPattern is a red herring: keys aren't patterns). Alternatively use ToString. But generally this just sounds like a dangerous and confusing game to play, to me. What exactly are you trying to do? There is probably a better way that doesn't involve using the keys of associations.

2. No, Map and AssociationMap shouldn't evaluate keys. Please report that behavior to tech support and it'll wend its way to Konstantin. If it's a simple fix it'll probably happen for 10.0.2, if it requires a big rewrite or a loss in efficiency it may take longer.

3. Keys have to remain unevaluated for associations to have any efficiency advantages over ordinary lists of rules. There's no way around that.

Answer 2

In 10.1.0 we have the new function KeyValueMap. You can use these functions, although they are not perfect

heldNormal[assoc_] := 
 Apply[Rule, Hold@Evaluate@KeyValueMap[Hold, assoc], {2}]
map[f_, assoc_] := 
 Association @@ 
  Unevaluated @@@ List@MapAt[f, heldNormal@assoc, {All, All, 2}]
associationMap[f_, assoc_] := 
 Association[ReleaseHold@Map[f, heldNormal[assoc], {2}]]

Example

With

testAssoc = Association[Unevaluated[{a -> b, c -> b}]];
b := (Print@#; #) &@"Fail";
a := (Print@#; #) &@"Boo";

We get

map[Hold, testAssoc]
associationMap[
 Function[Null, 
  Identity @@ RuleDelayed @@@ MapAt[Hold, Hold[#], {{1, 1}}], 
  HoldAll], testAssoc]

<|a->Hold[b],c->Hold[b]|>
<|Hold[a]:>b,Hold[c]:>b|>

Notes

Unfortunately heldNormal does not know whether to use Rule or RuleDelayed, which Normal does do. For this purpose, we could consider an alternative using something like the following snippet

Normal[Hold /@ KeyMap[Hold, testAssoc]]

Perhaps this answer can be extended with alternatives to other members of the *By family. See my answer here.