12

I just considered if/how one could implement Sequence in Mathematica if it were not predefined. It turned out that the following simple definition has in all my tests exactly the right behaviour:

myseq /: f_[x___, myseq[y___], z___] := f[x, y, z]

Now my question: Does this already correctly reproduce the full behaviour of Sequence, or is there something Sequence does but myseq doesn't which I missed in my tests?

Here's what I tested:

foo[myseq[a, b]]
(*
==> foo[a, b]
*)
Hold[mysec[a,b]]
(*
==> Hold[a, b]
*)
HoldComplete[myseq[a,b]]
(*
HoldComplete[myseq[a, b]]
*)
Hold[f[myseq[a,b]]]
(*
==> Hold[f[myseq[a, b]]]
*)
f[myseq[myseq[a,b],c,d],e,myseq[f,g,myseq[]]]
(*
==> f[a, b, c, d, e, f, g]
*)
celtschk
  • 19,133
  • 1
  • 51
  • 106
  • One situation where myseq fails is for something like {1, 2, 3} /. 2 -> myseq[a, b]. – Heike Jun 18 '12 at 18:57
  • t = {{a, b}, {c, d}, f}; myseq @@t fails – Dr. belisarius Jun 18 '12 at 18:58
  • 1
    @Heike That is because it doesn't support SequenceHold like Sequnce does. – Szabolcs Jun 18 '12 at 18:58
  • @Szabolcs I just figured that out. – Heike Jun 18 '12 at 18:58
  • @belisarius myseq[1,2] does the same. It may also be due to the lack of SequenceHold support. EDIT: it's not. Support can be added with myseq /: f_[x___, myseq[y___], z___] /; FreeQ[Attributes[f], SequenceHold] := f[x, y, z] – Szabolcs Jun 18 '12 at 19:00
  • Ah, so the answer to my question is "no". Looking at the attributes of Rule I notice an attribute with the suggestive name SequenceHold. Adding /; !MemberQ[Attributes[f], SequenceHold] to the definition seems to fix this problem. Any others? – celtschk Jun 18 '12 at 19:07
  • Is the code up to date? – Rojo Jun 18 '12 at 19:13
  • @belisarius: In which way does t = {...}; myseq @@t fail? I get the same result as with Sequence, except that of course I have a myseq instead of Sequence in the output. Is that what you considered "failing"? – celtschk Jun 18 '12 at 19:15
  • @Rojo: What do you mean? – celtschk Jun 18 '12 at 19:15
  • I mean, you've started suggesting changes to fix things in comments – Rojo Jun 18 '12 at 19:16
  • @celtschk myseq[1, 2] gives me an error about If being called with 6 arguments, possibly during the formatting step (ToBoxes) – Szabolcs Jun 18 '12 at 19:16
  • @Szabolcs: Ah, I didn't get those errors because I directly typed into the kernel. Now trying the notebook interface I also get those errors. The SequenceHold test doesn't fix it either. – celtschk Jun 18 '12 at 19:25
  • @Rojo: I didn't edit my post after initial submission, if that's what you mean. – celtschk Jun 18 '12 at 19:26
  • @celtschk I always bump into the problem of needing to Trace or Block the evaluation that happens during formatting, not evaluating the input expression. I always wanted to ask a question on this but I thought the answers wouldn't bring anything new ... maybe it's time to ask that question soon: in this case ToBoxes can't emulate the formatting step completely. – Szabolcs Jun 18 '12 at 19:32

3 Answers3

7

Ok, my two cents. The answer seems to be - you can't. There are 3 "magic" symbols which are wired into the core evaluator much deeper than the rest: Evaluate, Unevaluated, and Sequence. You can't fully emulate any of those without essentially writing your own version of Mathematica evaluator on top of the built-in one.

For the record, I first read about it in the book of David Wagner, "Power programming with Mathematica - the Kernel", p.207. Which means - if this is correct, I take the credit, but if it is wrong, he is the one to blame :). But, seriously, there was nothing in my experience to contradict this. You may emulate some aspects of Sequence, but I would be very surprised if you could make a complete emulation (without writing your own evaluator on top of the system one).

Let me also add that, while it is hidden, you do use Sequence in your approach, since the y___ pattern is internally destructured as Sequence. Check this out:

ClearAll[myseq];
myseq /: f_[x___, myseq[y___], z___] := f[x, Head[Unevaluated[y]], z]

and now

f[1, myseq[], 5]

f[1, Sequence, 5]
Leonid Shifrin
  • 114,335
  • 15
  • 329
  • 420
  • Leonid, I hate to disappoint but I think you're getting a false result in your test. I don't think ___ or ## are implemented using Sequence though they are of course analogous. In my opinion you cannot "catch" the y on the RHS in some intermediate form using Unevaluated. You are instead seeing this: Head@Unevaluated[1, 2, 3] yields Sequence; it is an effect of Unevaluated. – Mr.Wizard Jun 19 '12 at 01:30
  • @Mr.Wizard I'd have to sit down and experiment a bit to find where exactly, but I've seen __ and## transform into Sequence before. I think they do make use of Sequence at some point during their evaluation. – Szabolcs Jun 19 '12 at 07:02
  • @Mr.Wizard Well, you may be right, but I feel that Unevaluated should have nothing to do with that, it has a different purpose. OTOH, it is very natural that __ and ## use Sequence internally, since it is the same mechanism behind al of them. Perhaps, a better statement would be that all sequences are using the same underlying mechanism. I also second Szabolcs, I seem to remember cases where Sequence was appearing in the evaluation of __ and ## - related code. Will also try to find those cases. – Leonid Shifrin Jun 19 '12 at 07:53
  • 1
    @Mr.Wizard Ok, check this out: HoldComplete[1, 2, ff[3, 4, 5], 6, 7] /. ff[y__] :> y. – Leonid Shifrin Jun 19 '12 at 11:01
  • I almost second @Mr.Wizard except for the part where Unevaluated has a role (which I just don't have any hunch at all). I don't think Leonid's example proves that in his solution there's an internal use of Sequence. I imagine that when the replacer builds the expression :>rhs, if it a sequence, it has no other choice than to wrapping it in Sequence before putting it wherever it should be put. But that doesn't apply to his proposed solution, it's not a f[sth___]:=sth definition but something like ff[2, 3] /. ff[x___] :> HoldComplete[x]which seems to avoid Sequence – Rojo Jun 19 '12 at 14:32
  • Perhaps there's another internal sequence-like thing used that evaluates to Sequence in the end, but flattens itself out even in HoldComplete. No idea, but I think we are all guessing big about internals that probably won't ever be relevant :P – Rojo Jun 19 '12 at 14:33
  • @Rojo sorry, have you seen my last comment? If so, please tell me, how else than the way I did can you explain the result of this: HoldComplete[1, 2, ff[3, 4, 5], 6, 7] /. ff[y__] :> y. By your (or Mr.Wizard's) logic, we should get here HoldComplete[1,2,3,4,5,6,7], while we get HoldComplete[1,2,Sequence[3,4,5],6,7]. I'd rather think that his examples reveal some inconsistency in the pattern-matching / destructuring, than the other way around. – Leonid Shifrin Jun 19 '12 at 15:05
  • 1
    What would you have Head@Unevaluated[1,2] return, or y[x__] := x; y[2, 3] or {1, 2} /. {l__} :> l? Sequence is the standard way in MMA to represent sequences of arguments, and I see no other reasonable alternative. However, where there are alternatives such as in {1, 2} /. {l__} :> HoldComplete[l], l__ is shown to represent the "real" sequence and not a _Sequence expression – Rojo Jun 19 '12 at 15:47
  • @Rojo Ok, you may or may not be right. Pattern-matcher is sufficently separated from evaluation process so that this could be true, but also sufficiently integrated into it that it could be not. This is really a question of how deep the pattern-matcher is integrated with the evaluator, in this particular respect. – Leonid Shifrin Jun 19 '12 at 15:55
  • I agree 100% now. Answering your previous comment, I think of replace and family as something that works in 3 steps: match, build, replace. In the first 2, there are clearly deep internal things going on since __ matches "real" sequences. Same for the building. But the expression built has to be a MMA expression. – Rojo Jun 19 '12 at 16:03
5

I think an issue that you will never be able to completely solve is one of priorities. Sequence is flattened before upvalues. So, for starters

ClearAll[f];
f /: g[f, _] := 9

So

g[f, myseq[2, 3]]

9

but

g[f, Sequence[2, 3]]

g[f, 2, 3]

Rojo
  • 42,601
  • 7
  • 96
  • 188
2

There are functions related to Sequence: BlankSequence, BlankNullSequence and SlotSequence. These can be used to perform operations similar to Sequence but they are not identical.

As pointed out in the comments Sequence obeys the attribute SequenceHold:

SetAttributes[test, SequenceHold]

test[ Sequence[1, 2, 3] ]

test[Sequence[1, 2, 3]]

The others do not:

{1, 2, 3} /. _[x__] :> test[x]

test[##] &[1, 2, 3]

test[1, 2, 3]

test[1, 2, 3]

In fact these operations succeed even if test has HoldAllComplete.

You could emulate this behavior at least in part by modifying your definition to check for SequenceHold:

myseq /: f_[x___, myseq[y___], z___] /; 
  FreeQ[Attributes @ Unevaluated @ f, SequenceHold] := f[x, y, z]

Unlike the functions above Unevaluated uses Sequence:

test[1, Evaluate@Unevaluated[5, 6], 3]

test[1, Sequence[5, 6], 3]
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371