18

I have a question regarding the use of TikZ. I have five large sets of data (15000 points) and I want to use TikZ to plot my data, but obviously the amount of memory isn't sufficient. Can anyone give me any suggestions on how I can solve this problem? I have tried using epslatex with gnuplot but the thing is, I prefer to use TikZ since that way I don't have to worry about the fonts etc.

Roelof Spijker
  • 17,663
  • 5
  • 55
  • 63
Nil
  • 181
  • What kind of plot is that? Can you show us an output obtained via other software MATLAB, Mathematica, R etc. ? – percusse Feb 03 '12 at 17:42
  • If this is for a pseudo colour plot, or a scatter plot, then perhaps this or this may be useful. In both cases, the plot canvas is a PNG, and pgfplots is used to wrap it in nice axes etc. – qubyte Feb 03 '12 at 19:11
  • On another note, if this is a line plot(s) then pgfplots can be configured to read every nth point, effectively reducing your data set and the memory needed to contain the resultant plot. – qubyte Feb 03 '12 at 19:14
  • I recently had a dataset that was too large for pdflatex to handle. I escaped to lualatex where those memory limitations don't exist (afaik). I set the page margins to the one I use in my document, plotted the dataset and then used pdfcrop to create a pdf-picture of my plot. The upside is a small file-size, the downside is obviously the work you have to put in. Another option is, plot it with another software, remove the axis and include the picture in the plot, pgfplots supports graphics – Martin H Feb 06 '12 at 11:43
  • @Nil: Welcome to TeX.sx! I converted your "answer" post to a comment. Your initial session seem to have expired and your second post was therefore under a different unregistered account. This stopped you from adding a comment. I merged both accounts together now. Please consider registering your account to avoid such things in the future. Happy TeXing! – Martin Scharrer Feb 06 '12 at 12:24
  • @Nil: You should now be able to add an image because you have now over 15rep. – Martin Scharrer Feb 06 '12 at 12:26

1 Answers1

20

First off, if you are plotting 15,000 points one has to wonder what exactly you are trying to visualize. With so many points the "signal" or "point" of your graphic is likely to be lost in all the "noise".

Have you considered applying some sort of statistical analysis to your data in order to identify and emphasize trends? For example, this plot (produced using ggplot2) uses hexagonal binning to visualize a dataset containing 53,940 observations in a way that is much, much more effective than a scatterplot:

Hexbin Example

I want to use tikz to plot my data, but obviously the memory isn't enough.

If you absolutely have to plot a large dataset, don't use pdfTeX or XeTeX as your typesetting engine. Use the newer LuaTeX engine which is able to dynamically allocate memory.

LuaTeX can handle a scatterplot of 15,000 elements with no memory problems but it will take ~half a minute to typeset the plot. If you go this route, keep your graphics in external .tex files and include the .pdf images to keep the compile time low for your main document.

Sharpie
  • 12,844
  • 6
  • 48
  • 58
  • 1
    You can also just let R do the binning into hexagonal bins and let pgfplots do the actual plotting: http://tex.stackexchange.com/a/42442/2552 – Jake Feb 03 '12 at 21:37
  • Pgfplots is a great package. When working with R, another alternative is to save plots directly to TikZ using tikzDevice – Sharpie Feb 04 '12 at 04:54
  • 1
    Thanks for your answers! :) the picture is the result from 5 different simulations,I am performing a simulation on a transient process and each point on my figure represents a step in may calculations. the calculations go on for 15000 steps and therefore there are 15000 points of data, ... the result from gnuplot is a simple line graph, stress vs. strain Thanks in advance (P.S. apparently I cannot attach a picture as I don't have enough reputation points! :( ) – Nil Feb 06 '12 at 11:05
  • Apart from the eye-candy, what's the motivation for using hexagons rather than squares (easier to code/understand/turn into numbers)? – Federico Poloni Feb 06 '12 at 13:05
  • 1
    @FedericoPoloni See the vignette for the R package hexbin. Specifically, the "Theory and Algorithm" section explains why hexagons are superior to squares. – Sharpie Feb 06 '12 at 21:04
  • @Sharpie: thanks for the link, good explanation. For those too lazy to look it up, essentially the argument is "hexagons are visually less biased [...] [W]ith squares our eyes are drawn to the horizontal and vertical lines of the grid." – Federico Poloni Feb 07 '12 at 08:32
  • I've added some explanation how to produce a hexbin plot in another answer – Martin Thoma May 16 '13 at 12:35