I need to ray trace polychromatic light through an optical system that includes reflective diffraction gratings. I got stuck here.
If I have a reflecting plane with normal $\mathbf{\hat{n}}$ and an incident wave described by wave vector $\mathbf{k_i} = (2 \pi / \lambda) \ \mathbf{\hat{k}_i}$ where $\lambda$ is the wavelength and $\mathbf{\hat{k}_i}$ is the normal of the incident ray, I can get the wave vector of the reflected wave as
$$\mathbf{k_r} = \mathbf{k_i} - 2\mathbf{\hat{n}}(\mathbf{\hat{n}} \cdot \mathbf{k_i})$$
Here $(\mathbf{\hat{n}} \cdot \mathbf{k_i})$ is the component of $\mathbf{k_i}$ perpendicular to the surface and the $-2\mathbf{\hat{n}}$ factor "reflects it back up".
If I now add a periodic 1D grating of period $d$ to the surface such that the reciprocal vector of the grating $\mathbf{g}$ is in the plane ($|g| = 2 \pi/d$), the specularly reflected wave vector $\mathbf{k_r}$ is now also the 0th order diffracted wave vector.
But now I'd like to calculate vectors for the $\pm1$ orders by somehow adding $\pm\mathbf{g}$ and I'm not sure how to do this correctly.
Since diffraction doesn't change the wavelength, $|k_{+1}| = |k_{0}| = |k_{-1}| = |k_{i}|$.
Do I take the component of $\mathbf{k_r}$ parallel to $\mathbf{\hat{g}}$ (i.e. ($\mathbf{k_r} \cdot \mathbf{\hat{g}})$) and add $\pm \mathbf{g}$ to it, then just somehow foreshorten the components perpendicular to $\mathbf{g}$ to keep $|k_{+1}|$ and $|k_{-1}|$ equal to $|k_i|$?
If that's the right idea, and I'm not sure of that; how would that be written in vector form?
update:
I've gone through Born & Wolf and Eugene Hecht's Optics (and several other optics tomes nearby on the shelf) and was not able to find anything I can apply in this case. Textbooks like to stick to the simpler case where $\mathbf{g}$ is in plane of incidence, where the solutions reduce to the simple, familiar grating equation:
$$d(\sin \theta_i - \sin \theta_m) = m \lambda$$
Related in History of Science and Mathematics SE: