Basic approach:
A pure LaTeX solution could be some like:
\documentclass{article}
\begin{document}
Leslie Lamport was the initial developer of \LaTeX, a document preparation system\cite{lamport94} based on \TeX.
\begin{thebibliography}{999}
\bibitem{lamport94}
Leslie Lamport,
\emph{\LaTeX: A Document Preparation System}.
Addison Wesley, Massachusetts,
2nd Edition,
1994.
\end{thebibliography}
\end{document}

Although the environment thebibliography works perfectly and this could be all you need, my suggestion is to use this approach only if you have very few references that you'll never use again. Otherwise you will lose the great advantages of using BibTeX or biblatex.
BibTeX approach
When you have many references, it is worth to use BibTeX (the executable is bibtex in lowercase) instead of the embedded bibliographic system of LaTeX, so you can handle the references in a more efficient way. For example, you can select a subset of references without deleting unused references from a huge database than can be used in several documents. But more important, you have the control the final appearance of these references, how they are sorted and how are cited with only some commands in the LaTeX file, without modifying each reference by hand. May be for a publication with 200 references a editor will ask to use numeric citations sorted in order of appearance with the surnames of the authors before of the initials, but after that you want to publish the same text with another editor that demand references in alphabetic order, with the complete names before the surnames and with author-year citations like "Smith (2013)" instead of numbers in brackets as "[4]". Make this modifications manually in cites and references is a huge work and then BibTeX is essential.
For use BibTeX you need a .bib file like References.bib where the references are stored as plain text, with entries like this:
@ARTICLE{Gill,
author = "A. E. Gill",
title = "Some Simple Solutions for Heat-Induced Tropical Circulation",
journal = "Quart. J. R. Met. Soc.",
volume = 106,
year = 1980,
pages = "447-462",
}
You can write this file by hand using any text editor, but it is easier with an oriented program to manage BibTeX archives, as Jabref, Zotero or Mendeley.
Note that after @ARTICLE{ the word Gill is the key to cite the reference, equivalent lamport94 in the above example. It could be something different to the surname of the author, like Ref120 but obviously must be unique and preferably indicative, so usually the best is a combination of the first surname and year (Gill1980) and may be the first page (Gill1980p447) or some meaningful word of the title in order to avoid duplicate keys.
Once you have all the references in your References.bib file, you can include these keys in the a LaTeX file as follow:
\documentclass{article}
\begin{document}
Someone have found a simple solution \cite{Gill}
\bibliography{References}
\bibliographystyle{plain}
\end{document}
Then you can compile your document in this way:
pdflatex MyDocument.tex
bibtex MyDocument
pdflatex MyDocument.tex
pdflatex MyDocument.tex
And the result must be:

This is apparently a more complicated way to make the same, but if you simply change plain by vancouver, you will obtain the surname before of the initials and the journal without italics without touching References.bib. And if you use natbib in the preamble,
\usepackage{natbib}
then you can use styles as chicago to obtain non numeric citations, with more citations options. Example:
\documentclass{article}
\usepackage{natbib}
\begin{document}
\cite{Gill} have found a simple solution.
A simple solution have been found \citep{Gill}.
A simple solution have been found \citep[see page 47]{Gill}.
A simple solution have been found by \citet[in][see page 47]{Gill}.
\bibliography{References}
\bibliographystyle{chicago}
\end{document}
Note that now also the references changed (now the year is after the author also in references).

Biblatex & Biber approach
This newer approach can use the same database references that used in BibTeX "as is", but with more fields in each reference. For instance, BibTeX use the field year but biblatex can manage also exact dates in a date field.
In this approach you must use the package biblatex and different commands to link the .bib file and print the references in the main document. To compile it you can use also BibTeX with the option backend=bibtex, but by default is used another compiler: Biber (option backend=biber), that is a modern alternative with several advantages.
Obtain a numeric citation with biblatex is also fairly simple as you only need the package without options, and change \bibliography and \bibliographystyleof BibTex by \addbibresource{References.bib} (now with the extension) in the preamble and \printbibliography stating the obvious, where the bibliography must be printed.
\documentclass{article}
\usepackage{biblatex}
\addbibresource{References.bib}
\begin{document}
Someone have found a simple solution \cite{Gill}.
\printbibliography
\end{document}

For an author(year) citation style you simply need add the option style=authoryear and the usual \cite{} command, or \parencite{} for the (author, year) cites, but you can use now some others commands as \citeyear{}. Moreover, you can also add the option natbib=true to use natbib specific commands as \citep{}(nice if you are re-using some text having these commands).
\documentclass{article}
\usepackage[style=authoryear,natbib=true]{biblatex}
\addbibresource{References.bib}
\begin{document}
Someone have found a simple solution \parencite{Gill}.
Someone have found a simple solution \citep[see page 47]{Gill}.
A simple solution have been found by \citeauthor{Gill} in \citeyear{Gill}.
\printbibliography
\end{document}

The compilation must follow just the same 4 steps that above, but remember that now you should use biber instead of bibtex.
pdflatex MyDocument.tex
biber MyDocument
pdflatex MyDocument.tex
pdflatex MyDocument.tex
But why the hell use biblatex and biber instead of BibTeX? Well, although the result is not identical, the real advantage of biblatex cannot be appreciated in these simple examples. The power of biblatex is the great (and intimidating) flexibility (explained in a manual of 253 pages). For example, see Align/avoid numeration and labels in a custom biblatex format for a unusual format that hardly could be implemented in BibTeX nor with the thebibliography environment.
For a more detailed explanation, see also these great answers:
- What to do to switch to biblatex?
- bibtex vs. biber and biblatex vs. natbib
- Guidelines for customizing biblatex styles.
.bibfile (JabRef is a good option to deal with this type of files) that you can cite in a LaTeX file as showed here. You need to compile the LaTeX file withpdflatex file.tex, thenbibtex fileand againpdflatex file.tex(one or two times). Once you know what are you doing, search information in this site aboutbiblatex. – Fran Sep 19 '13 at 22:12