The trick is to store the list of all < and > appearing in the document inside the aux file, and use it in the second run to decide what should be < and >, and what should be \left\langle and \right\rangle.
The solution is a bit long, sorry. See the comments inline.
\makeatletter
% We could probably add some customization here.
%
% < and > as a relation or as a delimiter
\mathchardef\lt@relation=\mathcode`\<
\mathchardef\ltgt@bar@relation=\mathcode`\|
\mathchardef\gt@relation=\mathcode`\>
% Note: \ltgtstep and \ltgtunstep are needed because of the
% \left and \right. Also note that their placement (inside the group)
% is critical.
\def\lt@delimiter{\left\langle\ltgtstep}
\def\ltgt@bar@delimiter{\middle|}
\def\gt@delimiter{\ltgtunstep\right\rangle}
% < and > revert to the relation symbol if the .aux
% disagrees with what we see in the current run.
\let\lt@error@relation\lt@relation
\let\ltgt@bar@error@relation\ltgt@bar@relation
\let\gt@error@relation\gt@relation
\let\lt@error@delimiter\lt@relation
\let\ltgt@bar@error@delimiter\ltgt@bar@relation
\let\gt@error@delimiter\gt@relation
% As in other solutions, we make < and > active in math mode.
% The corresponding control sequences have been defined above.
\begingroup
\catcode`\<=\active
\catcode`\|=\active
\catcode`\>=\active
\gdef<{\lt@active}
\gdef|{\ltgt@bar@active}
\gdef>{\gt@active}
\endgroup
\mathcode`\<="8000
\mathcode`\|="8000
\mathcode`\>="8000
% ===================
% The above commands will be called via
% \csname lt@\ltgt@error@\ltgt@type\endcsname
% where \ltgt@error@ is {} by default and can be {error@}, and
% where \ltgt@type is {relation} or {delimiter}.
\gdef\ltgt@error@{}
\def\ltgt@type{relation}
% \ltgt@list will hold our list of "<", ">", "|". Each time we append,
% we check that the symbol agrees with the corresponding one from the
% .aux file. If not, we fall back on the error mode.
\gdef\ltgt@list{}
\def\ltgt@append#1{%
\xdef\ltgt@list{\ltgt@list#1}%
\ltgt@prev@pop%
\unless\ifx\ltgt@prev@head#1%
\gdef\ltgt@error@{error@}%
\fi%
}
% We pop the list from the .aux file as we construct the new list.
\def\ltgt@prev@pop{\expandafter\ltgt@prev@pop@aux\ltgtprevlist\relax\relax}
\def\ltgt@prev@pop@aux#1#2\relax{%
\xdef\ltgtprevlist{#2}%
\ifx#1\relax%
\global\let\ltgt@prev@head\relax%
\else%
\global\let\ltgt@prev@head#1%
\fi}
% To take care of grouping
\def\ltgt@append@open{%
\loop
\ifnum\ltgt@depth<\currentgrouplevel
\ltgt@append{(}%
\global\advance \ltgt@depth by 1\relax
\def\ltgt@type{relation}%
\repeat
}
\def\ltgt@append@close{%
\loop
\ifnum\ltgt@depth>\currentgrouplevel
\ltgt@append{)}%
\global\advance \ltgt@depth by -1\relax
\aftergroup\ltgt@append@close
\repeat}
% We will later \let<\lt@active and \let>\gt@active. For now, we just
% define these commands. Each has two pieces: first fill the \ltgt@list,
% putting many "|" to ensure that different groups are really separated
% by at least one "|". Second, typeset the correct symbol depending on
% what can be read from the prevlist (extracted from the .aux file).
\newcount\ltgt@depth
\def\ltgtstep{\global\advance\ltgt@depth by 1\relax}
\def\ltgtunstep{\global\advance\ltgt@depth by -1\relax}
\def\ltgt@openclose{%
\aftergroup\ltgt@append@close%
\unless\ifnum\ltgt@depth=\currentgrouplevel%
%\global\ltgt@depth\currentgrouplevel%
\ltgt@append@open%
\fi%
}
\def\lt@active{%
% fill the list
\ltgt@openclose%
\ltgt@append{<}%
% typeset
\ltgt@ifnextgt@TF{\def\ltgt@type{delimiter}}{\def\ltgt@type{relation}}%
\csname lt@\ltgt@error@\ltgt@type\endcsname%
}
\def\ltgt@bar@active{%
% fill the list
\ltgt@openclose%
\ltgt@append{|}%
% typeset
\csname ltgt@bar@\ltgt@error@\ltgt@type\endcsname%
}
\def\gt@active{%
% fill the list
\ltgt@openclose%
\ltgt@append{>}%
% typeset
\csname gt@\ltgt@error@\ltgt@type\endcsname%
\def\ltgt@type{relation}%
}
% When we see a <, we use \ltgt@ifnextgt@TF{.t.}{.f.} to test if the
% head of the prevlist is a >. If so, we execute {.t.}, otherwise {.f.}.
\newcount\ltgt@ifnextgt@count
\def\ltgt@ifnextgt@TF{%
\global\ltgt@ifnextgt@count0\relax%
\expandafter\ltgt@ifnextgt@readone\ltgtprevlist\relax\relax%
\expandafter\ltgt@ifnextgt@aux\ltgtprevlist\relax%
}
\def\ltgt@ifnextgt@readone#1{%
%\ltgt@ifnextgt@count
%\@ltgt@ifnextgt@false%
\ifx#1\relax
\expandafter\ltgt@ifnextgt@throw@FT
\fi
\ifx#1(%
\global\advance\ltgt@ifnextgt@count by 1\relax
\fi
\ifx#1)%
\global\advance\ltgt@ifnextgt@count by -1\relax
\fi
\ifnum\ltgt@ifnextgt@count<0\relax % no ">" can be found.
\expandafter\ltgt@ifnextgt@throw@FT
\fi
\ifnum\ltgt@ifnextgt@count=0\relax
\ifx#1<\relax % the first relevant character is not ">"
\expandafter\expandafter\expandafter\ltgt@ifnextgt@throw@FT
\else
\ifx#1>\relax % ">" was found!
\expandafter\expandafter\expandafter\expandafter
\expandafter\expandafter\expandafter\ltgt@ifnextgt@throw@TF
\fi
\fi
\fi
\ltgt@ifnextgt@readone
}
\def\ltgt@ifnextgt@throw@FT#1#{\ltgt@use@FT}
\def\ltgt@ifnextgt@throw@TF#1#{\ltgt@use@TF}
\def\ltgt@ifnextgt@aux#1#2#{% arg #2 delimited by brace, thrown away.
\ifx#1>%
\expandafter\ltgt@use@FTF%
\else%
\ifx#1|%
\expandafter\expandafter\expandafter\ltgt@use@T%
\else%
% \ifx#1(%)
% \expandafter\expandafter\expandafter\expandafter
% \expandafter\expandafter\expandafter\
\expandafter\expandafter\expandafter\ltgt@use@FFT%
\fi%
\fi%
{\ltgt@ifnextgt@aux#2}%
}
\long\def\ltgt@use@T#1{#1}
\long\def\ltgt@use@TF#1#2{#1}
\long\def\ltgt@use@FT#1#2{#2}
\long\def\ltgt@use@FTF#1#2#3{#2}
\long\def\ltgt@use@FFT#1#2#3{#3}
% We put the definition of the relevant list in the .aux file at the end
% of the run. This file is read at \begin{document}.
\AtEndDocument{\write\@auxout{\gdef\noexpand\ltgtprevlist{\ltgt@list}}}
%
% To ensure that \ltgtprevlist is defined in the first run, we do
\let\ltgtprevlist\relax
% Embedding the test example in the package itself (bad idea, but eh...)
\unless\ifx\documentclass\@twoclasseserror
\documentclass{article}
\begin{document}
The middle delimiter now works: $<a^2|x_{\sum_{i>j_1}i}>$, and nesting as well:
\[
<\frac{<u|v>+<v_1|v_2>}{<v_1|v>} v_1|v > = <u|v> + <v_1|v_2>,
\qquad {i<j}, {k>l}. % Note the use of braces to prevent seeing <j,k>.
\]
How come the end didn't become $i<j, k>l$? because I enclosed each
inequality in braces. Another case where this can be useful is to get
$<{<a,b>} a,b> = <a,b>^2$.
One more test:
\[
< {\sum |\lambda_i| v_i}| v > = 0
\]
\end{document}
\fi
:=one should really use\coloneqqfrommathtoolsbecause the output of\coloneqqis vertically symmetric. – Caramdir Aug 19 '10 at 10:09