If you have some background with the terminal/command line, then I would suggest the following tools.
Perhaps an overkill for many people, but how about python with numpy/scipy. And how about sage which is also built on top of python?
gnuplot is also a powerful free and open source graphing utility that can be used as a standalone application or with LaTeX packages pgfplots and tkz-fct. You can search this site for examples. For instance, the post: pgfplots: post-processing data created with gnuplot
R is a free and open source statistical software that is also widely used. It even has Sweave and knit-r packages just for running R commands in a TeX document. There have also been several examples in this site about these packages.
When I was using Windows, I remember also to have used winplot for simple graphs.
Most standalone graphics software has the ability to export into picture files. These can then be included in your LaTeX document with the \includegraphics command. For LaTeX graphing packages like pgfplots which is built on top of tikz on the other hand, you can write the code directly into your .tex document.
Update
I forgot to mention Scilab, which is also a free alternative to Matlab.
If you are looking for something to draw graphs in Graph Theory, you might want to look at tkz-berge which is also built on top of tikz. Here is an example from the manual.

\documentclass[10pt,tikz,convert,border=5]{standalone}
\usepackage{tkz-berge}
\begin{document}
\begin{tikzpicture}
\tikzset{VertexStyle/.style = {shape = circle,
shading = ball,
ball color = green,
minimum size = 24pt,
draw},
EdgeStyle/.style = {thick,
double = orange,
double distance = 1pt,
}}
\SetVertexLabel\SetVertexMath
\grCrown[RA=3,RS=6]{4}
\end{tikzpicture}
\end{document}