The four expression primitives (\numexpr, \dimexpr, \glueexpr, \muexpr) can be used anywhere that a register of the same type can be. This means that \numexpr produces integers, \dimexpr produces dimensions, \glueexpr produces glue expressions and \muexpr produces muskip values. As such, something like the example in the question
\the\dimexpr 12pt / 2.3pt\relax
cannot yield a real number as the result has to be a dimension. (As we'll see below, the result here is 6pt followed by .3pt\relax.) (I've added \the here so you can typeset the result rather than having to use it in an assignment. I'll do that throughout.)
In all cases, the formal syntax is that the expressions absorb tokens and carry out appropriate mathematics up to either a \relax or the first non-valid token in the input. This is easiest to see with \numexpr, which must produce an integer. So for example
\the\numexpr 2 + 2pt
will print 4pt, which is made up of the result of the expression (4) followed by input which cannot be part of the expression and thus terminates it. This is useful in comparisons, where the comparator will necessarily terminate the expression
\ifnum\numexpr ... > \numexpr ... \relax
In the case of \dimexpr and \glueexpr, the formal syntax requires that the input consists of a dimension (or glue value) followed by an optional multiplication factor, and not the other way around. The factor may itself be a numerical expression. Thus all of
\the\dimexpr 1pt + 1pt\relax
\the\dimexpr 1pt * 10\relax
\the\dimexpr (1pt + 1pt) / (1 + 1)\relax
are valid and give the expected results, but
\the\dimexpr 10 * 1pt\relax
is not valid and leads to an error.
Expressions may use the operators +, -, * and / along with parenthesis (/). Division takes place with rounding, and this applies to any number expressions inside dimension/glue expressions. Thus for example we get
\the\dimexpr 5pt * (3 / 2) \relax % 10pt
\the\dimexpr 5pt * (4 / 3) \relax % 5pt
In glue expressions the plus/minus parts do not need parenthesis to be affected by a factor as they are necessarily one entity. So for example
\the\glueexpr 5pt plus 1pt * 2 \relax
yields 10pt plus 2pt.
Within expressions, TeX will coerce other numerical types into the correct form in the same way as occurs when doing register assignment. So
\the\numexpr\dimexpr 1pt\relax\relax
will result in 65536, i.e. the value of 1pt converted to scaled points and then coerced into an integer. With \glueexpr here the stretch/shrink would be dropped. Going the other way, any \numexpr inside a \dimexpr or \glueexpr will need appropriate units providing
\the\dimexpr\numexpr 1 + 2\relax pt\relax
\numexpr \dimexpr12pt / \dimexpr2.3pt. But you get 5, while you probably wanted 5.2, since the division rounds to the nearest integer. (Not to be confused with \divide, which truncates instead of rounding.) Did you check out section 3.5 of the manual? In any case, if you want real division, there exist packages that do what you want. – Mark May 19 '15 at 02:30\dividewith a dimension but in\numexpr \dimexpr12pt / \dimexpr2.3ptTeX coerces the dimensions into integers in scaled points then does the number expression division (which rounds as already noted). – Joseph Wright May 19 '15 at 07:27