21

I have data in file.txt which can be expressed by 20000x11 matrix. I do not want to import all of these data. How can I plot graph of the column 1 & column 4?

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
Gokce
  • 211
  • 1
  • 2
  • 3
  • 3
    You will anyway have to import some data. If the built-in Import is too slow for you, you can try setting up a custom importer function similarly to how it was done here. – Leonid Shifrin May 22 '12 at 16:48

3 Answers3

24

If the data are comma-delimited and saved as myfile.txt then

data =Import["C:\\Users\\md\\Desktop\\myfile.txt", {"Data",  {All}, {1, 3}}];

imports columns 1 and 3, giving:

(*{{"a1", "a3"}, {"b1", "b3"}, {"c1", "c3"}, {"d1", "d3"}}*)

For comparison,

Import["C:\\Users\\md\\Desktop\\myfile.txt", {"Data"}]

gives

(*{{"a1", "a2", "a3", "a4", "a5"}, {"b1", "b2", "b3", "b4", 
  "b5"}, {"c1", "c2", "c3", "c4", "c5"}, {"d1", "d2", "d3", "d4", 
  "d5"}} *)

More information may be found here ("How to Import a Spreadsheet")

user1066
  • 17,923
  • 3
  • 31
  • 49
  • I had forgotten about sub-elements. +1 – rcollyer May 23 '12 at 00:35
  • 1
    What if it's space delimited? Are there options or do I have to convert my data file? – Malabarba Nov 30 '12 at 00:11
  • 1
    @BruceConnor Good question. As far as I can see, space delimited still works Import["C:\\Users\\md\\Desktop\\myfileSpace.txt", {"Data", {All}, {1, 3}}], for example. It would be good to know if you have got it to work. – user1066 Nov 30 '12 at 19:33
  • Yes, I got it to work. – Malabarba Dec 03 '12 at 13:49
  • What if I want to plot the second column in this data? how can i do that? –  May 16 '13 at 20:22
  • @suzy. I am not sure I have fully understood your question. However... If, say, data = {{0, 0, 0}, {1, 20, 100}, {2, 40, 200}, {3, 60, 300}} and I would like to plot column 1 versus colum 2 AND (on the same graph) column 1 versus columm 3, I could do that as follows: ListLinePlot[ Transpose /@ {{data[[All, 1]], data[[All, 3]]}, {data[[All, 1]], data[[All, 2]]}}, PlotMarkers -> Automatic, PlotStyle -> {Red, Green}]. Perhaps you were thinking of something different to ListLinePlot? – user1066 May 17 '13 at 12:17
7

If the file is very large and importing it takes too much memory, you can use external tools to cut the columns you are interested in. You can often call these tools directly from Mathematica, for example

ReadList["!awk '{print $1, $4}' file.txt", {Number, Number}]

This will use awk to cut columns 1 and 4. ReadList, with an explicit specification of the type of each column, is typically going to be faster and more memory efficient than Import.

This approach can be significantly faster than Import["file.txt", "Table"][[All, {1,4}]] if the file is large enough, and especially if it contains heterogeneous data which Import must auto-detect.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • Assuming *nix environment? Nice to see awk in use - been a while :) +1 for ! form which i didn't know about. – SEngstrom May 17 '13 at 00:58
  • @SEngstrom Yes, assuming Unix for the '-quoting to work. I'm not sure how to specify an awk program on the command line on Windows---I think it should be "-quoted. It should work on Windows too (if awk is installed and in the path). I used Windows until recently. The !-syntax works in practically all importing functions and it's quite useful – Szabolcs May 17 '13 at 01:57
3
data=Import["myFile","Table"][[All,{1,4}]];
ListPlot@data
acl
  • 19,834
  • 3
  • 66
  • 91
image_doctor
  • 10,234
  • 23
  • 40
  • 3
    Unfortunately, that does not adhere to the OPs request to not import all of the data. – rcollyer May 22 '12 at 17:51
  • @rcollyer I can't disagree with you :) But I sense some ambiguity in the original question. The data isn't too large, it should fit in memory with ease and may well produce the result the OP wanted. I'll happily withdraw the answer if it isn't what is required. – image_doctor May 22 '12 at 18:01
  • Let's see what they say. If it is on a memory constrained machine, then sure, but as you said, it is easily loadable in memory. Rough estimate, with 64 bit (8 byte) real numbers: 1.7 MB. Right. +1 – rcollyer May 22 '12 at 18:17
  • @rcollyer that's what I thought, no problem, we'll leave it up to to them:) – image_doctor May 22 '12 at 18:30