Sum all columns from a list

Question

I have a list of column names:

list = {"col1", "col2", "col3"}

and a dataset ds. I want to add a new column to ds that contains the sum of the columns listed in list. As suggested here, I could do:

ds[All, <| #, "colnew" -> #col1 + #col2 + #col3 |> &]

But I cannot list the column names by hand, because the list may be very large, or I might not know the names of the columns a priori.

WReach · Accepted Answer · 2017-06-01T16:02:17.530

7

If the you want to total all columns instead of just those on a select list, then the suggestion given by @Kuba is appropriate:

ds[All, <| #, "colnew" -> Total@# |> &]

If you wish to total only the listed columns, then:

ds[All, <| #, "colnew" -> Total[#[[list]]] |> &]

For example:

list = {"col1", "col2", "col3"};

ds = AssociationThread[list->#]& /@ RandomInteger[10, {4, 3}] // Dataset

ds[All, <| #, "colnew" -> Total[#[[list]]] |> &]

Or, asking for an actual subset of columns:

list2 = {"col1", "col2"};

ds[All, <| #, "colnew" -> Total[#[[list2]]] |> &]

edited Jun 01 '17 at 16:02

answered Jun 01 '17 at 15:46

WReach

68,832
4
164
269

I suppose I missed the point :) – Kuba Jun 01 '17 at 16:02
@Kuba Actually, I just reworded my response to emphasize your suggestion -- to match the title of the post :) – WReach Jun 01 '17 at 16:02
I don't sure you have read my answer here.But I have read yours many many times,which make me upset sometimes I have to say.I think your solution works just beacuse result by a unexpected behavior derive from MMA.Note this two f is in different place(one is in the inner of Association，but another out of it,and I think it is should in outside).Though they is a same operation.I think the solution is right,I mean that behavoir is expected. – yode Jul 02 '17 at 23:09
@yode You are right -- the documentation makes no explicit mention of this syntax. But it does say that a list of rules will be treated as if the rules has been specified directly. Associations are treated like lists in many functions, so it would be surprising if Association itself did not do that. I think the syntax is a logical consequence of these principles. For the first example in your screenshot to work, I would expect f to evaluate to a rule, list or association. – WReach Jul 03 '17 at 05:23

score 3 · Answer 2 · answered May 31 '17 at 22:46

3

You can programmatically generate <| #, "colnew" -> #col1 + #col2 + #col3 |> & as follows.

list = {"col1", "col2", "col3"};

Tr[Slot /@ list] /. x_ :> (<|#, "colnew" -> x|> &)

Association[#1, "colnew" -> #col1 + #col2 + #col3] &

answered May 31 '17 at 22:46

Mr.Wizard

271,378
34
587
1,371

yode · Answer 3 · 2017-06-16T15:13:03.447

1

ds = Dataset[{<|"col1" -> 1, "col2" -> 2|>, <|"col1" -> 3, 
    "col2" -> 4|>, <|"col1" -> 5, "col2" -> 6|>}]

Mathematica graphics

ds[All, Append[#, "colnew" -> Total[#]] &]

Mathematica graphics

edited Jun 16 '17 at 15:13

answered Jun 16 '17 at 12:25

yode

26,686
4
62
167

Sum all columns from a list

3 Answers3

Linked